root/sp
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity
96 Commits 8 Branches 0 Tags
cf7a6b910c006bcd4e0aa521ca63ac160c0b72ab
Commit Graph

83 Commits

Author SHA1 Message Date
Semih
b38721bb05 feat(api): POST /api/insights/[id]/status (status + optional notes) 2026-05-14 11:58:30 +00:00
Semih
a713505f44 feat(insights): Telegram alerts + daily brief push notifications 
apps/worker/src/lib/telegram.ts:
- sendTelegram() with Redis-backed dedupe (NX SETEX, 1h TTL)
- Helpers: alertP0Insight, alertRegression, alertSanitizationAnomaly, alertBudgetCap

Wired into:
- analyze.ts: P0/P1 insight creation → instant alert (dedupe per insight_id);
  budget guard halt → daily cap alert (dedupe per state per day)
- validation.ts: regression detected (≥3 sessions w/ same fingerprint after shippedAt)
  → alert (dedupe per insight_id)
- compress-sessions.ts: sanitization anomaly (>500 tokens, 0 PII matches)
  → alert (dedupe per session_id) — possible PII leak warning

Daily Brief (jobs/daily-brief.ts):
- Cron @05:00 UTC (= 08:00 Europe/Istanbul)
- 24h: sessions/insights/cost/cache-hit + 3 top priorities + 7d shipped/validated/regressed
- POST /api/insights/brief/send for manual trigger / smoke test

Env: TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, PANEL_PUBLIC_URL (Coolify both apps).
Bot: @Pl24_mitm_bot (AiFactory), chat 7840804807. Source: airflow3 monitoring DAG.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 11:36:10 +00:00
Semih
74f0ff4935 fix(ingest): rolling lookback + ongoing-aware watermark to prevent missed sessions 
Bug: previous logic advanced watermark to the latest session start_time including
ongoing sessions. PostHog session_recordings filters by start_time, so once a
session was 'seen' as ongoing the watermark moved past its start time and the
session was never re-fetched after it ended. Today 4 auth sessions on
/dashboard/vehicles/* and /dashboard/search (07:17-07:40 UTC) were lost this way.

Fix:
1. ROLLING_LOOKBACK_MINUTES (default 60): every cycle queries date_from =
   min(watermark, now - 60min). Sessions that just finished get re-fetched
   regardless of watermark drift. Upsert dedupes.
2. Track earliestOngoingStart; cap watermark to (earliestOngoingStart - 1s)
   so subsequent cycles re-read that range.

Also added GET/DELETE /api/insights/watermark for manual reset (used to
trigger 24h backfill after this deploy).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 10:53:38 +00:00
Semih
dfc19c8f13 refactor(integrations): migrate issue tracker GitHub → Gitea 
- apps/web/src/lib/gitea.ts: createIssue / getIssue / verifyWebhookSignature / buildIssueBody
  * Endpoint: git.semih.ai/api/v1 (configurable via GITEA_BASE_URL)
  * Auth: 'Authorization: token <PAT>' (Gitea convention)
  * Labels: Gitea expects numeric IDs not strings → ensureLabels() resolves/creates
    with color coding (P0/P1 red, P2 yellow, P3 green, type-* grey, default blue)
  * Webhook signature: X-Gitea-Signature (hex, no sha256= prefix)
- apps/worker/src/lib/gitea.ts: read-only getIssue() for sync polling
- _actions.ts + github-sync.ts now import from /lib/gitea
- Removed old apps/{web,worker}/.../lib/github.ts + /api/webhooks/github route
  (the receiver was already dead — sp.semih.ai is Tailscale-only)
- UI: 'GitHub' label → 'Gitea' on insight detail card
- github-sync job filters by githubIssueUrl.startsWith(GITEA_BASE_URL) so legacy
  GitHub-hosted insights (semihyesilyurt/sase.tr#20) stay frozen rather than
  collide with same-numbered Gitea issues at root/sase.tr.

Env migration (Coolify, panel-web + panel-worker):
- removed: GITHUB_TOKEN, GITHUB_REPO_SASE, GITHUB_WEBHOOK_SECRET
- added:   GITEA_TOKEN, GITEA_REPO_SASE=root/sase.tr, GITEA_BASE_URL=https://git.semih.ai

Provisioned Gitea PAT 'super-panel-insights' (scopes: write:repository + write:issue),
stored in Bitwarden.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 09:36:57 +00:00
Semih
d0c497d12d feat(panel-taxonomy): track Faz 2 sase.tr events (page_visibility, idle, shortcuts, referrals, settings, bulk) 
7 new event names added to TRACKED_EVENTS so enrich.ts fetches them for compression
timeline interleaving + tagger context. No new tag rules yet (Faz 2 is enrichment,
not signal extraction).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 06:28:20 +00:00
Semih
b83d28c119 feat(eval): POST /api/insights/eval-sets thin wrapper (create + optional run) 
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 06:15:39 +00:00
Semih
b635b529e2 feat(phase6e): eval framework + prompt editor + daily brief + retention + bundle mode 
Schema:
- eval_sets (promptTag, cases JSON: [{id, timeline, expected: {...}}])
- eval_runs (per-execution scoring: passedSchema/Severity/Rubric, cost, tokens, results)

Prompts:
- 2 new seed templates: upgrade_hesitation (flash), pattern_bundle (pro)
- pickPromptTag() routes 'upgrade_hesitation' tag to upgrade_hesitation prompt
- Editor UI at /insights/settings/prompts/[id]: edit system/user/schema/tier/temp,
  publishPromptVersion() creates new version + deactivates old (CRUD with auto bump)
- setPromptActive() to toggle versions

Daily Brief (/insights/brief):
- Last 24h: sessions processed, insights produced (with severity breakdown),
  cost, cache hit rate; today's top 5 priorities; week stats (shipped/validated/regressed)

Retention cron (04:15 UTC daily):
- Delete sessions_meta + session_custom_events older than 90d (unless referenced by
  active insight)
- Delete compressed_sessions rows + MinIO timeline blobs older than 180d
- raw_metadata 30d (currently no-op; we don't persist raw metadata to MinIO)

Eval framework:
- apps/worker/src/jobs/eval-run.ts: runs cases against current/specified prompt version,
  scores schema_pass + severity_match + rubric_substring; stores EvalRun
- apps/worker BullMQ queue handler for 'eval-run' job name
- apps/web installed bullmq; /lib/queue.ts thin Queue accessor
- Web actions: createEvalSet, triggerEvalRun (enqueues job to insight-pipeline queue)
- UI: /insights/settings/eval-sets list, /new create form (paste JSON cases),
  /[id] detail with Run button + recent runs + per-case JSON

Bundle mode (analyze job):
- Pull 3x batch, group by fingerprint
- Groups ≥ INSIGHT_BUNDLE_THRESHOLD (default 3) → use 'pattern_bundle' prompt
- Timeline = primary rep + PATTERN BUNDLE summary block (occurrences, unique users, deltas)
- Insight stores ALL group session IDs as relatedSessionIds; all marked analyzed in one tx
- Cost amortized: 1 LLM call per group

Nav/Cmd+K:
- Inbox header links: Brief, Patterns, Eval sets
- Palette: Daily brief, Eval sets entries

Deferred to backlog: embedding-similarity cross-fingerprint clustering, Telegram brief delivery,
Sase.tr-side data-private audit (separate repo).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 06:11:48 +00:00
Semih
9c7682ae63 feat(phase6d): GitHub polling sync — outbound-only state mirror (Tailscale-only panel) 
Webhook can't reach sp.semih.ai from internet (Tailscale-only DNS). Switched to a
worker job that polls GitHub for tracked issue states every 10 minutes.

apps/worker/src/lib/github.ts: read-only getIssue() client
apps/worker/src/jobs/github-sync.ts: scan insights with githubIssueNumber + not validated/dismissed/duplicate,
  fetch remote state, reflect transitions:
    open → closed = shipped + shippedAt + validationStartedAt
    closed → open = in_progress + clear validation state

Scheduler: github-sync@*/10min added.

The /api/webhooks/github route stays (works if reachable in future) but is no longer
the source of truth.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 05:54:58 +00:00
Semih
0c6350db2d feat(insights): POST /api/insights/[id]/create-issue thin wrapper around server action 
Useful for programmatic smoke tests and future automation. Auth-gated via underlying action.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 05:49:20 +00:00
Semih
c5acdfc8ec feat(phase6d): GitHub action loop — issue creation, webhook, validation cron, patterns view 
Schema:
- Insight.+githubIssueNumber (user-visible #N, separate from id BigInt)

GitHub integration (apps/web/src/lib/github.ts):
- repoFor(projectKey): env-based GITHUB_REPO_<KEY>=owner/repo mapping
- createIssue / getIssue REST wrappers
- verifyWebhookSignature (HMAC-SHA256 timing-safe)
- buildIssueBody: renders structured markdown from insight + LLM body
  (hypothesis, reproduce steps, affected route/provider, quick/long fixes,
  suggested investigation, evidence links to panel, DoD checklist)

Server action createGithubIssueForInsight:
- Auth-gated, audited; idempotent (refuses if issue already exists)
- Labels: insight-driven, severity-<P>, type-<T>, <project>-pilot
- Sets status=in_backlog, stores githubIssueUrl/Id/Number/State

Webhook /api/webhooks/github:
- Signature verify with GITHUB_WEBHOOK_SECRET
- issues.closed → status=shipped + shippedAt + validationStartedAt
- issues.reopened → status=in_progress + clear validation state
- issues.opened → status=in_backlog
- PR linking placeholder (passthrough only for now)

Validation cron (worker, daily 5:00 UTC):
- For each insight in 'shipped' state:
  - Count sessions with same fingerprint after shippedAt
  - >= INSIGHT_REGRESSION_THRESHOLD (default 3) → status=regressed + regressionDetected=true
  - validationPeriodDays elapsed with no regression → status=validated + validatedAt

UI:
- Insight detail: GithubActions card — Create button (when no issue),
  external link + issue # + state (when present)
- New /insights/patterns page: clusters insights by type + affected_route/provider,
  shows ≥2-insight or ≥5-occurrence groups sorted by max severity
- Inbox header link to Patterns
- Cmd+K palette: Patterns entry

Env needed: GITHUB_TOKEN, GITHUB_REPO_SASE, GITHUB_WEBHOOK_SECRET.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-14 05:45:48 +00:00
Semih
5beb89f771 feat(phase6c): insight inbox actions + filters + keyboard shortcuts + budget UI + prompt registry view 
Server actions (apps/web/src/app/insights/_actions.ts):
- setStatus / bulkSetStatus (11 allowed states)
- setFounderNotes / setFounderPriority / setSeverityOverride
- updateBudgetSetting (key whitelist)
All audit-logged + revalidate /insights paths.

Inbox enhancements:
- Filter bar: severity / status (active default = new+triaged+in_backlog+in_progress+regressed)
  / type chips with live counts, query-param driven
- Per-row action buttons: Backlog / Defer / Dismiss with toast feedback
- Keyboard shortcuts (j/k navigate, Enter open, b backlog, e defer, d dismiss, ? help)
  Skipped when input/textarea focused.

Insight detail (/insights/i/[id]):
- FounderForm component: 6 status buttons, severity override dropdown,
  founder_priority pin input, founder_notes textarea (saved separately)

Budget settings (/insights/settings/budgets):
- Editable rows for monthly/daily caps, per-call max, min_score_for_analysis,
  cache_ttl_hours, analysis_paused (kill switch)

Prompt registry (read-only, full editor in 6e):
- /insights/settings/prompts list
- /insights/settings/prompts/[id] detail w/ system prompt, user template,
  output schema, recent cost ledger rows for this template

Cmd+K palette extended with Insights group:
- Inbox, Cost dashboard, Pipeline, Budget settings, Prompt registry

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 23:32:41 +00:00
Semih
0295b88981 fix(phase6b): DeepSeek V4 disable thinking by default + parse text block (not first content) 
- thinking={type:disabled} skips reasoning trace → output_tokens drop 10x
- find first content block of type='text' (V4 may emit thinking before text)
- DEEPSEEK_THINKING=true env to re-enable if needed for hard problems

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 23:11:09 +00:00
Semih
95c7e8e321 feat(phase6b): LLM analysis layer — DeepSeek + insights + cost ledger + budget guard 
Schema:
- insights (fingerprint dedup, occurrence_count, related_session_ids[], priority_score)
- cost_ledger (per-call, cache_hit/miss/output token split + USD)
- prompt_templates (versioned, per-tag, with JSON schema + tier + temperature)
- budget_settings (global key/value: monthly_hard_cap_usd, daily_soft/hard, per_call_max, analysis_paused)

Worker:
- lib/deepseek.ts: Anthropic-compat endpoint client (deepseek-v4-flash | deepseek-v4-pro),
  usage→USD with V4 promo pricing, cache_read_input_tokens awareness, extractJson() helper
- lib/budget.ts: checkBudget() returns active|soft_throttled|hard_paused|monthly_paused,
  forces flash tier on soft cap, halts on hard/monthly cap or analysis_paused
- lib/prompts.ts: 5 seed templates (bug_triage P, ux_friction F, payment_issue P,
  onboarding_stuck F, provider_quality F) with embedded JSON schemas + Sase.tr context;
  pickPromptTag() maps session tags → prompt
- lib/json-validate.ts: lightweight schema validator (no ajv dep)
- lib/seed-runtime.ts: idempotent upsert of prompts + default budget settings on worker boot
- lib/minio.ts: +getText() for compressed timeline fetch
- jobs/analyze.ts: budget guard → 6h fingerprint cache (attach session to existing insight) →
  template lookup → severity-based tier override → DeepSeek call → JSON parse + validate →
  insert insight (or aggregate occurrence) → write cost_ledger
- scheduler: analyze@*/4min on insight-pipeline queue

UI:
- /insights (was pipeline view) → now Insight Inbox: priority-sorted list w/ KPI strip
  (new/in_backlog/shipped/today $/month $), severity badges, link to detail
- /insights/i/[id]: insight detail with structured body render, related sessions,
  per-session cost breakdown, raw JSON collapsible
- /insights/costs: KPI cards (today, month, avg, cache hit), daily 30d bar table,
  by-model + by-prompt breakdowns, top 10 expensive, recent errors
- /insights/pipeline: moved old session-pipeline view here
- /insights/sessions/[id]: unchanged session timeline viewer

Defaults:
- monthly cap $30, daily soft $1.50 / hard $3, per-call $0.20, analysis_paused=false
- Severity→Tier: P0/P1=pro, P2/P3/INFO=flash; budget soft-cap forces flash

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 23:01:55 +00:00
Semih
978b71d818 fix(phase6a-v1.2): PostHog event filter client-side + backfill distinct_id + power_user_path rrweb fallback 
- listSessionEvents: PostHog only honors single event= param; switched to client-side filter
- tag-sessions: backfill posthogDistinctId from getRecording when null (for rows pre-column)
- tagger: power_user_path v1 fallback (auth + clicks>=20 + no errors), vin_decode_no_outcome P3
- ingest upsert: always update posthogDistinctId (idempotent)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 22:43:44 +00:00
Semih
7606e1b0bf feat(insights): /api/insights/reprocess auth-gated endpoint to re-run pipeline on existing sessions 
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 22:26:13 +00:00
Semih
fc6a3f7a39 feat(phase6a-v1.2): dual-mode custom-event enrichment (v1.0 + v2.0 aliases) 
- Prisma: posthog_person_cache (24h TTL), posthog_group_cache, session_custom_events
- SessionMeta: +posthogDistinctId, +groupKey, +customEventCount
- PostHog client: listSessionEvents, getPerson, getGroup
- posthog-cache.ts: cache-with-stale-refresh for person + group properties
- event-taxonomy.ts: dual-mode alias map (vin_decoded↔vin_decode_initiated,
  vin_decode_success↔vin_decode_succeeded, vin_decode_error↔vin_decode_failed,
  payment_success↔payment_succeeded). 56 tracked event names total.
- enrich.ts: per-session custom events fetch, persist to session_custom_events,
  attach user_properties + group_properties via cache.
- tagger.ts: 14 new custom-event tags
  (vin_decode_fail_pattern, provider_reliability_issue, payment_friction,
   payment_ui_silent_failure (P0), payment_failed_session, checkout_abandonment,
   downgrade_pending, compatibility_quality_gap, parts_export_abandoned,
   search_validation_friction, onboarding_stuck, upgrade_hesitation,
   power_user_path, at_risk_active_session, provider_mismatch,
   api_key_friction, webhook_setup_struggle); group-context priority bump.
- heuristic.ts: customEventPromoteReasons() — 10 custom-event promote signals.
- compress.ts: merge_rrweb_and_custom_events interleaving with ⤷ EVENT: prefix,
  ±1.5s grouping, standalone flush for events with no nearby rrweb. Adds
  company_context header line; key_events lists top-5 custom event counts.
- /insights page: +Events column. /insights/sessions/[id]: collapsible custom
  events table + group key display.

Graceful: when Sase.tr emits 0 custom events the pipeline falls back to
rrweb-only tagging from Phase 6a v1.1.

PRD: behavioral-insight-pipeline-prd-1.md (v1.2),
     sase-posthog-events-prd.md (Sase.tr-side, separate codebase).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 22:17:18 +00:00
Semih
93499faeaa fix(phase6a): PostHog snapshot blob fetch uses start_blob_key/end_blob_key; parse [window_id,event] tuples 
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 21:26:36 +00:00
Semih
795528982e fix(phase6a): increase BullMQ lockDuration to 5min (network/db calls exceed 30s) 
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 21:15:06 +00:00
Semih
1a7a1b5787 feat(phase6a): behavioral insight pipeline ingestion + tagging + compression 
- prisma: sessions_meta, compressed_sessions, ingestion_watermarks
- worker: PostHog client (eu.i.posthog.com), heuristic filter
- worker: BullMQ insight-pipeline queue (ingest 5min / tag 2min / compress 3min)
- worker: tagger (bug_suspected, ux_friction, upgrade_hesitation, etc.) + severity scoring
- worker: rrweb -> semantic timeline transform + 8-pattern PII sanitization
- worker: fingerprint hash + MinIO upload (bucket: insight-compressed)
- web: /insights pipeline dashboard + session timeline viewer
- sidebar: Insights nav entry

LLM (Phase 6b), insight inbox, cost dashboard, GitHub loop deferred to later phases.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-13 20:58:36 +00:00
Semih
31040a76b4 feat(phase5): coolify ops + backup status dashboard 
- lib/coolify.ts typed client (whitelisted apps)
- /operations: deploy/restart server actions per app, audited
- /operations: backup status cards reading MinIO listings
- lib/minio.ts shared client
- docs/phase5-deferred.md (migration runner + stripe webhook rationale)
 2026-05-13 11:22:59 +00:00
Semih
909cacf7b3 feat(phase4): rate limit + audit archive + panel backup + DR docs 
- src/lib/rate-limit.ts (Redis sliding window, fail-open)
- before-hook on /sign-in/email: 5 attempts/min per IP+email
- worker job audit-archive (daily 03:30, JSONL → MinIO, 90d retention)
- worker job panel-backup (daily 04:00, pg_dump -Fc -Z9 → MinIO)
- Dockerfile adds postgresql16-client
- scripts/restore-drill.sh restores latest dump into panel_drill
- docs/disaster-recovery.md + docs/phase4-deferred.md (mTLS + Infisical rationale)
 2026-05-13 11:14:36 +00:00
Semih
ed79f4eacd feat(phase3c): SSE /api/events/stream + live /events page 2026-05-13 11:06:32 +00:00
Semih
b9709ed07e feat(phase3): worker container + event bus + scheduled jobs 
apps/worker:
- BullMQ nightly scheduler (cron 0 3 * * *)
- Redis Streams consumer-group per wired/active project
- Persists events to Event model

schema:
- Event model (streamId unique, project + type indexed)
 2026-05-13 11:03:58 +00:00
Semih
e455ebbfce feat(phase2): planned badge, Cmd+K palette, per-project dashboard cards 
- Project.status (planned/wired/active) + seed updates
- ProjectBadge component
- CommandPalette (⌘K) wired in PanelShell with project + nav + sign-out
- Dashboard '/' shows per-project cards + KPI strip
- Added doner312 to seeded projects
 2026-05-13 10:54:07 +00:00
Semih
7b83ef8f11 feat(phase1d): sase admin-sdk skeleton + endpoint inventory on /projects/sase 2026-05-13 10:43:07 +00:00
Semih
e4e734fc66 feat(phase1c): audit sign-in / sign-out via better-auth after hook 2026-05-13 10:40:26 +00:00
Semih
f72dcc05ba feat(phase1b): sase.tr read-only connection 
- prisma/sase/schema.prisma with User + UserSubscription (partial view)
- lib/db-sase.ts exports typed RO client
- /projects/sase shows live metrics (users count, new users 7d, active subs)
- seed marks sase as active

DATABASE_URL_SASE_RO wired to super_panel_reader (SELECT-only)
 2026-05-13 10:38:03 +00:00
Semih
d189206f06 feat(phase1a): admin-sdk base, audit helper, project seed, panel pages 
- src/lib/audit.ts + src/lib/admin-sdk/base.ts (typed HTTP + auto audit)
- prisma/seed.ts seeds 5 projects (all inactive)
- pages: /projects, /projects/[key], /audit, /operations, /events, /settings
- components/panel-shell.tsx shared layout
- entrypoint runs seed after db push
 2026-05-13 10:33:18 +00:00
Semih
5aca3934ca feat: shadcn/ui + sidebar-07 + dashboard-01 layout 
- shadcn init (neutral base, css variables)
- AppSidebar wired to panel routes (Overview/Projects/Operations/Events/Audit/Settings)
- NavUser dropdown uses Better Auth signOut
- SectionCards reads live counts from panel DB
- TooltipProvider wraps root layout
 2026-05-13 10:12:57 +00:00
Semih
b2f59420b5 fix: secure cookie name in middleware; drop TOTP (tailscale gates access) 2026-05-13 10:04:52 +00:00
Semih
d906a6488f chore: disable signup — single-user panel 2026-05-13 09:33:02 +00:00
Semih
db890c3e18 fix: dockerfile build flow + drop standalone 2026-05-13 09:27:58 +00:00
Semih
67a7c5b887 feat: phase 0 skeleton — next.js 16 + better-auth + prisma 2026-05-13 09:17:50 +00:00