feat(insights): multi-project Sentry archive

The sentry-archive job was hardcoded to a single SENTRY_PROJECT slug.
Now that sase has a second Sentry project (sase-web, browser SDK), the
worker has to pull both — otherwise frontend events live only in
Sentry's UI and never reach the panel's sentry_events table for the
behavioral-insight pipeline to pick up.

Changes:

- SENTRY_PROJECT becomes a CSV (e.g. "python,sase-web") parsed once at
  module load into PROJECTS[]. Backward-compatible: a single value
  keeps working exactly as before.
- Issues are fetched org-wide in a single loop (unchanged endpoint)
  but each issue's Sentry project slug is now persisted via the new
  `sentrySourceProject` column on sentry_issues.
- Events are project-scoped on Sentry's side, so the job iterates
  projects and calls listEventsPage(slug, cursor) for each. Per-project
  pagination + dedup is preserved. A project-level failure is recorded
  in `error` but doesn't abort the other projects.
- New result shape: `perProject: { [slug]: { fetched, inserted, duplicate } }`
  Pipeline log gains a `[python=N/M sase-web=N/M]` breakdown so it's
  obvious at a glance which project produced what.
- Schema: nullable `sentrySourceProject` on both SentryIssue and
  SentryEvent (+ composite index per projectKey for queries that want
  to filter "panel project = sase AND sentry project = sase-web").
  Migration is a pure nullable add — `prisma db push` on container
  start is safe. Existing rows stay null until next sync touches them.

After merge, set `SENTRY_PROJECT=python,sase-web` on the panel-worker
Coolify app and redeploy.
This commit is contained in:
Semih
2026-06-01 18:37:10 +03:00
parent 928728dc66
commit 8a81cb77bf
4 changed files with 147 additions and 74 deletions

View File

@@ -514,44 +514,48 @@ model PosthogCohortSnapshot {
// Sentry issue (group) aggregate state. Upserted to the latest snapshot — the
// raw history lives in SentryEvent; the issue row is the current grouping/metadata.
model SentryIssue {
id String @id // Sentry issue/group id
projectKey String // "sase"
shortId String?
title String?
culprit String?
level String?
status String?
type String?
count Int? // total events as of lastSyncedAt
userCount Int?
firstSeen DateTime?
lastSeen DateTime?
permalink String?
metadata Json?
ingestedAt DateTime @default(now())
lastSyncedAt DateTime @default(now())
id String @id // Sentry issue/group id
projectKey String // "sase" — panel-side project key
sentrySourceProject String? // Sentry project slug (e.g. "python", "sase-web") — distinguishes browser vs server errors when one panel project has multiple Sentry projects
shortId String?
title String?
culprit String?
level String?
status String?
type String?
count Int? // total events as of lastSyncedAt
userCount Int?
firstSeen DateTime?
lastSeen DateTime?
permalink String?
metadata Json?
ingestedAt DateTime @default(now())
lastSyncedAt DateTime @default(now())
@@index([projectKey, lastSeen])
@@index([projectKey, sentrySourceProject, lastSeen])
@@map("sentry_issues")
}
// Individual Sentry events (the at-risk raw occurrences; free tier prunes ~30d).
// Lossless full payload as JSONB; also cold-dumped to MinIO.
model SentryEvent {
eventId String @id // Sentry event id (32 hex)
issueId String? // parent issue/group id
projectKey String
title String?
message String?
level String?
platform String?
timestamp DateTime // event occurrence time
tags Json?
payload Json // full event payload — lossless
ingestedAt DateTime @default(now())
dumpedAt DateTime?
eventId String @id // Sentry event id (32 hex)
issueId String? // parent issue/group id
projectKey String // panel-side project key (e.g. "sase")
sentrySourceProject String? // Sentry project slug the event came from ("python", "sase-web")
title String?
message String?
level String?
platform String?
timestamp DateTime // event occurrence time
tags Json?
payload Json // full event payload — lossless
ingestedAt DateTime @default(now())
dumpedAt DateTime?
@@index([projectKey, timestamp])
@@index([projectKey, sentrySourceProject, timestamp])
@@index([issueId, timestamp])
@@index([dumpedAt])
@@map("sentry_events")