perf(proxy): capture/scrape browser'larında bant genişliği israfını kes
Some checks failed
QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled
Some checks failed
QA Gate (P0/P1) / Test affected app (pull_request) Has been cancelled
DataImpulse residential kotası hızlı tükeniyordu (15 günde 58.5 GB). CSV analizi capture/scrape tarayıcılarının tam sayfa (resim/font/CSS/reklam/ CDN) yüklediğini gösterdi. - pcat + emex capture: resource-blocking'e stylesheet + üçüncü-parti çöp (yastatic, google fonts/autofill, adsco.re, displayvertising, tidio, ipify) eklendi. Widget runtime CDN'leri (jsdelivr/unpkg) bilinçli hariç. - emex: engelleme context seviyesine alındı → tüm tab'ları kapsıyor. - pcat: ipify exit-IP probe artık PCAT_EXIT_IP_PROBE ile default-off (15 günde ~54k faturalı istek + 352 MB, sadece telemetri içindi). Not: emex scraping resmi tarayıcıda yüklemiyor; URL'yi HTML'den parse edip boyutu ayrı Range GET ile alıyor → resim engellemek scraping'i etkilemez. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -260,6 +260,42 @@ export class EmexBrowserService implements OnModuleInit, OnModuleDestroy {
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
|
||||
});
|
||||
|
||||
// Block heavy resources to save proxy bandwidth. Scraping reads HTML/DOM
|
||||
// only — schema-image URLs are parsed from the markup and their dims come
|
||||
// from a separate Range GET, so the browser never needs images/fonts/CSS.
|
||||
// Context-level route covers every tab (session establish + acquirePage).
|
||||
await this.context.route("**/*", (route) => {
|
||||
const url = route.request().url();
|
||||
const type = route.request().resourceType();
|
||||
|
||||
if (["image", "font", "stylesheet", "media"].includes(type)) {
|
||||
return route.abort();
|
||||
}
|
||||
|
||||
// Third-party junk seen in proxy usage: analytics, ad networks, chat
|
||||
// widgets, Google/Yandex assets, and our own ipify egress probe.
|
||||
if (
|
||||
url.includes("google-analytics.com") ||
|
||||
url.includes("googletagmanager.com") ||
|
||||
url.includes("mc.yandex.ru") ||
|
||||
url.includes("yastatic.net") ||
|
||||
url.includes("fonts.googleapis.com") ||
|
||||
url.includes("fonts.gstatic.com") ||
|
||||
url.includes("content-autofill.googleapis.com") ||
|
||||
url.includes("facebook.net") ||
|
||||
url.includes("doubleclick.net") ||
|
||||
url.includes("hotjar.com") ||
|
||||
url.includes("adsco.re") ||
|
||||
url.includes("displayvertising.com") ||
|
||||
url.includes("tidio.co") ||
|
||||
url.includes("api.ipify.org")
|
||||
) {
|
||||
return route.abort();
|
||||
}
|
||||
|
||||
return route.continue();
|
||||
});
|
||||
|
||||
this.startedAt = Date.now();
|
||||
this.sessionExpiry = 0; // force session establish on first acquirePage
|
||||
|
||||
|
||||
Reference in New Issue
Block a user