fix(insights): sanitizer warning — raise threshold + add real leak signal
The previous 500-token threshold fired on virtually every authenticated session because rrweb masks inputs at source and stripQuery() removes URL params before sanitize() runs — so 0 matches is the normal case, not an anomaly. Confirmed on session 019e455e: 9m32s authed session with ~1093 tokens, 0 matches, no actual PII leak. - Raise broad threshold 500→2000 tokens - Add precise leak detector: any unmasked \`input [tag] "text"\` line in the compressed timeline AND authed AND 0 matches → alert. This catches the actual failure mode (PostHog source-side masking broken) instead of guessing from total token volume. - Sync UI warning threshold to match (500→2000).
This commit is contained in:
@@ -73,9 +73,19 @@ export async function runCompressSessions(): Promise<{ compressed: number; faile
|
||||
customEvents,
|
||||
);
|
||||
|
||||
// Sanitization sanity check: large token output but zero matches is an anomaly.
|
||||
// Don't abort here (Phase 6a defers LLM), but flag it.
|
||||
const anomaly = out.tokenCountEstimate > 500 && out.sanitization.matches === 0;
|
||||
// Sanitization sanity check.
|
||||
// Most sessions legitimately have 0 PII matches — rrweb masks inputs at
|
||||
// source and stripQuery() removes URL params before sanitize() runs.
|
||||
// Two anomaly modes worth alerting on:
|
||||
// 1) very large output with 0 matches (covers all leaks broadly)
|
||||
// 2) any unmasked input ("text") slipped past PostHog masking
|
||||
// Mode 2 is the precise leak signal — unmasked inputs in the compressed
|
||||
// text mean the source-side mask failed. Format is `input [tag] "text"`.
|
||||
// Masked rows look like `input [tag] (masked)`.
|
||||
const unmaskedInputCount = (out.timeline.match(/^\d\d:\d\d → input [^\n]*?"[^"]/gm) ?? []).length;
|
||||
const anomaly =
|
||||
(out.tokenCountEstimate > 2000 && out.sanitization.matches === 0) ||
|
||||
(unmaskedInputCount > 0 && out.sanitization.matches === 0 && s.isAuthenticated);
|
||||
|
||||
const minioKey = `sase/${dateFolder(s.startedAt)}/${s.id}.txt`;
|
||||
await putText(COMPRESSION_BUCKET, minioKey, out.timeline);
|
||||
@@ -109,7 +119,9 @@ export async function runCompressSessions(): Promise<{ compressed: number; faile
|
||||
});
|
||||
|
||||
if (anomaly) {
|
||||
console.warn(`[compress] sanitization-anomaly session=${s.id} tokens=${out.tokenCountEstimate}`);
|
||||
console.warn(
|
||||
`[compress] sanitization-anomaly session=${s.id} tokens=${out.tokenCountEstimate} unmasked_inputs=${unmaskedInputCount}`,
|
||||
);
|
||||
void alertSanitizationAnomaly({
|
||||
sessionId: s.id,
|
||||
tokens: out.tokenCountEstimate,
|
||||
|
||||
Reference in New Issue
Block a user