perf(proxy): capture/scrape browser'larında bant genişliği israfını kes
Some checks failed
QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled

DataImpulse residential kotası hızlı tükeniyordu (15 günde 58.5 GB). CSV
analizi capture/scrape tarayıcılarının tam sayfa (resim/font/CSS/reklam/
CDN) yüklediğini gösterdi.

- pcat + emex capture: resource-blocking'e stylesheet + üçüncü-parti çöp
  (yastatic, google fonts/autofill, adsco.re, displayvertising, tidio,
  ipify) eklendi. Widget runtime CDN'leri (jsdelivr/unpkg) bilinçli hariç.
- emex: engelleme context seviyesine alındı → tüm tab'ları kapsıyor.
- pcat: ipify exit-IP probe artık PCAT_EXIT_IP_PROBE ile default-off
  (15 günde ~54k faturalı istek + 352 MB, sadece telemetri içindi).

Not: emex scraping resmi tarayıcıda yüklemiyor; URL'yi HTML'den parse edip
boyutu ayrı Range GET ile alıyor → resim engellemek scraping'i etkilemez.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-15 18:24:22 +03:00
parent a6437dcd67
commit 0f2126d694
2 changed files with 62 additions and 6 deletions

View File

@@ -260,6 +260,42 @@ export class EmexBrowserService implements OnModuleInit, OnModuleDestroy {
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
});
// Block heavy resources to save proxy bandwidth. Scraping reads HTML/DOM
// only — schema-image URLs are parsed from the markup and their dims come
// from a separate Range GET, so the browser never needs images/fonts/CSS.
// Context-level route covers every tab (session establish + acquirePage).
await this.context.route("**/*", (route) => {
const url = route.request().url();
const type = route.request().resourceType();
if (["image", "font", "stylesheet", "media"].includes(type)) {
return route.abort();
}
// Third-party junk seen in proxy usage: analytics, ad networks, chat
// widgets, Google/Yandex assets, and our own ipify egress probe.
if (
url.includes("google-analytics.com") ||
url.includes("googletagmanager.com") ||
url.includes("mc.yandex.ru") ||
url.includes("yastatic.net") ||
url.includes("fonts.googleapis.com") ||
url.includes("fonts.gstatic.com") ||
url.includes("content-autofill.googleapis.com") ||
url.includes("facebook.net") ||
url.includes("doubleclick.net") ||
url.includes("hotjar.com") ||
url.includes("adsco.re") ||
url.includes("displayvertising.com") ||
url.includes("tidio.co") ||
url.includes("api.ipify.org")
) {
return route.abort();
}
return route.continue();
});
this.startedAt = Date.now();
this.sessionExpiry = 0; // force session establish on first acquirePage