fix(insights): sanitizer warning — raise threshold + add real leak signal

The previous 500-token threshold fired on virtually every authenticated
session because rrweb masks inputs at source and stripQuery() removes
URL params before sanitize() runs — so 0 matches is the normal case,
not an anomaly. Confirmed on session 019e455e: 9m32s authed session
with ~1093 tokens, 0 matches, no actual PII leak.

- Raise broad threshold 500→2000 tokens
- Add precise leak detector: any unmasked \`input [tag] "text"\` line
  in the compressed timeline AND authed AND 0 matches → alert. This
  catches the actual failure mode (PostHog source-side masking broken)
  instead of guessing from total token volume.
- Sync UI warning threshold to match (500→2000).
This commit is contained in:
Semih
2026-05-20 17:07:57 +03:00
parent 42096814e3
commit 4984190273
2 changed files with 17 additions and 5 deletions

View File

@@ -73,9 +73,19 @@ export async function runCompressSessions(): Promise<{ compressed: number; faile
customEvents,
);
// Sanitization sanity check: large token output but zero matches is an anomaly.
// Don't abort here (Phase 6a defers LLM), but flag it.
const anomaly = out.tokenCountEstimate > 500 && out.sanitization.matches === 0;
// Sanitization sanity check.
// Most sessions legitimately have 0 PII matches — rrweb masks inputs at
// source and stripQuery() removes URL params before sanitize() runs.
// Two anomaly modes worth alerting on:
// 1) very large output with 0 matches (covers all leaks broadly)
// 2) any unmasked input ("text") slipped past PostHog masking
// Mode 2 is the precise leak signal — unmasked inputs in the compressed
// text mean the source-side mask failed. Format is `input [tag] "text"`.
// Masked rows look like `input [tag] (masked)`.
const unmaskedInputCount = (out.timeline.match(/^\d\d:\d\d → input [^\n]*?"[^"]/gm) ?? []).length;
const anomaly =
(out.tokenCountEstimate > 2000 && out.sanitization.matches === 0) ||
(unmaskedInputCount > 0 && out.sanitization.matches === 0 && s.isAuthenticated);
const minioKey = `sase/${dateFolder(s.startedAt)}/${s.id}.txt`;
await putText(COMPRESSION_BUCKET, minioKey, out.timeline);
@@ -109,7 +119,9 @@ export async function runCompressSessions(): Promise<{ compressed: number; faile
});
if (anomaly) {
console.warn(`[compress] sanitization-anomaly session=${s.id} tokens=${out.tokenCountEstimate}`);
console.warn(
`[compress] sanitization-anomaly session=${s.id} tokens=${out.tokenCountEstimate} unmasked_inputs=${unmaskedInputCount}`,
);
void alertSanitizationAnomaly({
sessionId: s.id,
tokens: out.tokenCountEstimate,