fix(pcat): widen JWT capture for slow residential IPs (domcontentloaded + 12s poll)
Some checks failed
QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

After the Floxy account rotation, the new residential exit IPs are slower:
measured DOM-ready ~7s and the widget's /v3/api/proxy token call firing ~11.8s —
past the 10s PAGE_TIMEOUT + networkidle, so every capture timed out ("No token
after 12s") and the warm pool stayed empty (pcat dead despite valid creds).

- waitUntil networkidle → domcontentloaded (reliable ~7s; networkidle often never
  settles on JS catalog sites through a slow proxy → goto times out pre-token).
- PAGE_TIMEOUT 10s → 25s (env PCAT_PAGE_TIMEOUT_MS); token fired at ~11.8s.
- CAPTURE_POLL_AFTER_OK 5s → 12s (env PCAT_CAPTURE_POLL_OK) to cover the ~5s
  gap between DOM-ready and the token call. Capture is background (warm pool,
  cold-pool fast-fail never blocks users), so the longer cap is free insurance.

Verified via standalone Playwright probe through the new Floxy: token captured at
11842ms (DOM 7023ms). typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-11 16:59:17 +03:00
parent 2e6c8ac751
commit 4a6e673af1

View File

@@ -36,16 +36,19 @@ const REDIS_SAFETY_BUFFER_S = 60; // don't serve from Redis if <60s of life left
const TOKEN_TTL = 600; // seconds — TWS- has no built-in expiry, refresh aggressively
const REFRESH_BUFFER = 90; // Refresh 90s before expiry
const CAPTURE_POLL_INTERVAL = 500; // ms
// Two-armed post-goto poll: a healthy goto means the widget JS already loaded
// and the API call typically fires within ~1-2s — short cap is enough. A
// failed goto means we lost the page, but the widget request may still have
// gone through before the navigation timeout — give a brief grace window
// and bail. Old single-armed 20s cap was the dominant cost on failed-site
// attempts (e.g. 10s page timeout + 20s blind poll = 30s wasted).
const CAPTURE_POLL_AFTER_OK = 10; // 10 × 500ms = 5s after a healthy goto
const CAPTURE_POLL_AFTER_FAIL = 4; // 4 × 500ms = 2s grace after goto failure
const PAGE_TIMEOUT = 10_000; // 10s — healthy partner sites load in <5s through
// the proxy; a longer wait only prolongs dead-port connects in capture retries.
// Post-goto poll. We now navigate with `domcontentloaded` (reliable, doesn't
// depend on networkidle settling on JS-heavy catalog sites), then poll for the
// widget's token call. Measured through Floxy residential 2026-06-11: DOM ready
// ~7s, the /v3/api/proxy token call fires ~11.8s (≈5s after DOM). The poll must
// comfortably cover that gap, so 12s — too short and every capture misses the
// token (seen at the old 5s cap: "No token after 12s", whole pool starved).
const CAPTURE_POLL_AFTER_OK = Number(process.env.PCAT_CAPTURE_POLL_OK) || 24; // 24 × 500ms = 12s
const CAPTURE_POLL_AFTER_FAIL = 8; // 8 × 500ms = 4s grace after goto failure
// 25s — residential exit IPs are slow; the token call fired at ~11.8s in the
// 2026-06-11 measurement, so 10s timed out before it. Capture is background
// (warm pool, never blocks a user with cold-pool fast-fail), so a longer cap is
// cheap insurance against residential variance. Env-tunable for future rotation.
const PAGE_TIMEOUT = Number(process.env.PCAT_PAGE_TIMEOUT_MS) || 25_000;
const CONTEXT_CLOSE_TIMEOUT = 5_000;
const SITE_COOLDOWN = 10 * 60 * 1000; // 10 min per site
const MAX_POOL_SIZE = 5;
@@ -763,12 +766,16 @@ export class PartsCatalogsAuthService implements OnModuleInit, OnModuleDestroy {
return route.continue();
});
// Navigate — networkidle waits for widget JS to load + make API calls
// Navigate with domcontentloaded (not networkidle): on JS-heavy catalog
// sites through a slow residential proxy networkidle often never settles
// and the goto times out before the widget fires its token call. DOM-ready
// is reliable (~7s measured); the request listener + post-goto poll then
// catch the /v3/api/proxy token call (~11.8s measured).
let gotoOk = true;
try {
await page.goto(siteUrl, {
timeout: PAGE_TIMEOUT,
waitUntil: "networkidle",
waitUntil: "domcontentloaded",
});
} catch (navErr) {
// Navigation may timeout but JWT could still be captured if the