Semih Yesilyurt
3514272936
EMEX leaf parts fetch was averaging 12.5s cold per category (Honda Civic
prod traces). Almost all of that lived in the scraper's two waterfall page
loads, each of which (a) waited for `networkidle` — which only fires after
analytics/ads stop chattering — then (b) slept an unconditional 2 seconds.
The actual extraction work runs in <100ms once the DOM is parsed.
Three focused changes in scripts/emex-vin-scraper.js:
1) getParts() resolves the Unit.aspx URL via a plain `fetch()` of
QuickDetails.aspx instead of opening it in the browser. Probe showed
the page is fully server-rendered and the anchor is present in the
initial HTML, so the browser stage is dead weight (~3s saved). Falls
back to a browser load if the fetch fails or the anchor isn't there,
so catalogs that gate the link behind JS still work.
2) Both remaining `page.goto()` calls in getParts/getCategories/
getCategoryTree drop `waitUntil:'networkidle'` for `'domcontentloaded'`
plus a targeted `waitForSelector('img.dragger' | 'a[href*=…]', {timeout:5000-8000})`.
`.catch(()=>null)` makes the wait advisory — the evaluate() below has
its own null-safe fallbacks — but in practice the selector is present
well before networkidle would have fired.
3) Removes the two unconditional `setTimeout(r,2000)` sleeps in getParts.
They predate the selector-wait pattern and were belt-and-braces.
Expected: ~12.5s → ~3-5s cold leaf fetch. Warm path (DB-cached) is
unchanged at ~45ms. Dev-only push for verification before main merge.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sase
VIN/şase numarası sorgulama ve otomotiv yedek parça katalog platformu (Türkiye). Site: https://sase.tr · Staging: https://dev.sase.tr
Stack
| Katman | Teknoloji |
|---|---|
| Monorepo | pnpm 10 + Turborepo |
| Backend | NestJS 10, TypeScript 5.7, Node 22 |
| DB | PostgreSQL 17 + Drizzle ORM |
| Cache / Queue | Redis 7 + BullMQ |
| Auth | Better Auth (cookie session) |
| Frontend | Vite 6, React 19, TanStack Router + Query |
| UI | Tailwind v4, shadcn/ui (Radix) |
| Ödeme | Stripe (kart) |
| Postal | |
| Storage | MinIO (S3) |
| Analitik | PostHog |
| Test | Vitest, Playwright |
| Deploy | GitHub Actions → SSH → PM2 |
Yapı
ss/
├── apps/
│ ├── api/ NestJS (port 4000, /api)
│ └── web/ Vite + React (port 3000)
├── packages/
│ ├── shared/ Tipler, Zod şemaları
│ ├── config/ Env şeması (Zod)
│ └── ui/ Paylaşılan shadcn bileşenleri
├── docker/ docker-compose + nginx
├── scripts/ Deploy + bakım scriptleri
└── docs/ Detay dökümantasyon (INDEX.md)
Komutlar
pnpm dev # tüm app'ler (Turbo)
pnpm build
pnpm test # Vitest
pnpm lint # Biome
pnpm typecheck
# DB (apps/api/)
pnpm db:push # şemayı push
pnpm db:studio # Drizzle Studio
pnpm db:generate # migration üret
# Tek app
pnpm dev --filter=api
pnpm dev --filter=web
VIN Decode Akışı
Corgi (offline WMI) → PartsCatalogs API → PL24 API → EMEX scraper → NHTSA
Birden fazla araç eşleşmesi gelirse frontend seçim modalı gösterir.
Katalog Mimarisi
- PL24 P5 Modern (REST/JSON) — VW Group, BMW, Mercedes, Renault, Toyota...
- PL24 P4 Legacy (HTML scrape) — Ford, PSA, Hyundai/Kia, Nissan, Opel, Volvo
- PartsCatalogs (REST + Playwright JWT) — geniş VIN kapsamı
- EMEX (Playwright scrape,
emexdwc.ae) — async BullMQ worker
EMEX & PartsCatalogs Türkçeleştirme
EMEX ve PartsCatalogs kaynakları kategori/parça isimlerini İngilizce (zaman zaman Rusça) döner. Çeviri akışı:
TranslationsService(apps/api/src/translations/) — Redis cache → DB lookup (emex_category_translations) → orijinal döndür. Dictionary tabanlı word-by-word replace devre dışı (yarı-İngilizce çıktı yaratıyordu).- Hot path (
categories.service.ts) — EMEX/PCAT insert noktalarındatranslateMany()ile bulk lookup yapılır; cache miss varsa orijinal yazılır, sonraki bootstrap pass'inde LLM ile çevrilir. - Bootstrap script (
scripts/emex-translate-bootstrap.ts) — DB'deki tüm uniquename_originaldeğerlerini OpenRouter üzerinden DeepSeek V3 ile çevirir,emex_category_translationstablosuna yazar. - Backfill script (
scripts/emex-backfill-tr-names.ts) — translation tablosuna göre mevcutcategories.nameveparts.namedeğerlerini günceller, Redis cache flush eder.
# 1) Tüm yeni terimleri çevir (~$0.50, ~30 dk)
pnpm exec tsx scripts/emex-translate-bootstrap.ts --dry-run
pnpm exec tsx scripts/emex-translate-bootstrap.ts
# 2) DB kayıtlarını güncelle
pnpm exec tsx scripts/emex-backfill-tr-names.ts --dry-run
pnpm exec tsx scripts/emex-backfill-tr-names.ts
# 3) API restart (yeni hot-path translateMany için)
pm2 restart sase-api
Env: OPENROUTER_API_KEY (apps/api/.env).
Test Bilgileri
- Admin:
admin@sase.tr/Sase2026 - Test VIN (VW):
WVWZZZ1JZ3W597935 - Login:
POST /api/auth/sign-in/email→ cookiebetter-auth.session_token
Daha Fazla
docs/INDEX.md— kapsamlı proje rehberi (rotalar, API, DB şeması, bileşenler)CLAUDE.md— Claude Code için proje rehberi