feat(sase): VIN decode anomaly detection + Telegram alerts

Adds the active half of Phase 8: dashboard was passive, this pushes
when something breaks. Targets MTTD < 10 minutes from the PRD.

Detection (panel-web)
- detectVinAnomalies() compares a 15-minute current window against a
  baseline built from the same 15-minute slot across the previous 7
  days (same-hour-of-day, average across days that had ≥ 1 sample).
- Five hits, two severities:
  * success_rate_drop — high at ≥ 5pp drop, critical at ≥ 15pp
  * p95_latency_spike — high at 2× baseline (and > 1s), critical at 3×
  * volume_drop — high at ≥ 80% below baseline, critical at ≥ 95%
  * volume_spike — high at ≥ 5× baseline
  * timeout_dominance — high when ≥ 50% of failures are timeouts
- Minimum-volume guards on both current and baseline so quiet hours
  don't generate noise (MIN_CURRENT_VOLUME=5, MIN_BASELINE_VOLUME=10).
- Each anomaly carries a 15-min-bucket dedupe key — same anomaly type
  fires at most once per bucket regardless of cron cadence.

Endpoint
- GET /api/internal/vin-anomaly-check, gated by x-internal-worker-token
  header (constant-time compare against INTERNAL_WORKER_TOKEN env).
  Returns { current, baseline, anomalies[] }.

Worker
- New job vin-anomaly-detect, BullMQ scheduler */5 * * * *.
  Fetches the panel endpoint, then for each anomaly calls
  alertVinAnomaly() — sendTelegram with the bucket dedupe key, so
  Redis SETEX NX dedupes across the 1h cooldown window.
- alertVinAnomaly() in worker/lib/telegram.ts formats the message with
  severity icon + baseline/observed/volume context + dashboard link.
- runVinAnomalyDetect returns { ok, checked, anomalies, alertsFired,
  alertsDeduped }; pipeline logs only when something happened or the
  check failed.

Env
- INTERNAL_WORKER_TOKEN set on both panel-web and panel-worker
  (32-byte hex, generated in Coolify).
- PANEL_INTERNAL_URL on panel-worker → coolify-network UUID hostname
  for panel-web, no Cloudflare/Tailscale hop on internal calls.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Semih
2026-05-18 12:22:02 +03:00
parent 66a5828ee6
commit 4694c1ffdb
5 changed files with 417 additions and 1 deletions

View File

@@ -0,0 +1,25 @@
import { NextResponse } from "next/server";
import { timingSafeEqual } from "node:crypto";
import { detectVinAnomalies } from "@/lib/sase/vin-anomaly";
export const dynamic = "force-dynamic";
const INTERNAL_TOKEN = process.env.INTERNAL_WORKER_TOKEN ?? "";
function validToken(req: Request): boolean {
if (!INTERNAL_TOKEN) return false;
const provided = req.headers.get("x-internal-worker-token") ?? "";
if (!provided) return false;
const a = Buffer.from(provided);
const b = Buffer.from(INTERNAL_TOKEN);
if (a.length !== b.length) return false;
return timingSafeEqual(a, b);
}
export async function GET(req: Request) {
if (!validToken(req)) {
return NextResponse.json({ ok: false, error: "unauthorized" }, { status: 401 });
}
const result = await detectVinAnomalies();
return NextResponse.json({ ok: true, ...result });
}