feat(FN-335): log coordination check-in — confirmed human-blocked at Step 1
Commits merged: - chore(FN-335): log coordination check-in — confirmed human-blocked at Step 1 Files changed: docs/product/funnel-audit-p0-cro-2026-05.md | 1 + 1 file changed, 1 insertion(+) Fusion-Task-Id: FN-335
This commit is contained in:
739
docs/product/funnel-audit-p0-cro-2026-05.md
Normal file
739
docs/product/funnel-audit-p0-cro-2026-05.md
Normal file
@@ -0,0 +1,739 @@
|
||||
# PostHog Funnel Audit — P0 CRO Sprint (FN-203, FN-199, FN-278)
|
||||
|
||||
**Date:** 2026-05-13
|
||||
**Auditor:** Fusion agent (FN-334)
|
||||
**Status:** ⏳ AWAITING MANUAL DATA — PostHog API key unavailable from worktree; human must collect data from PostHog dashboard at https://eu.posthog.com/project/127747
|
||||
**Data collector:** FN-336 (Fusion executor) — attempted, blocked. Human intervention required.
|
||||
**Coordinator:** FN-335 — verified blocked 2026-05-13. Awaiting human to complete Section 8 → Section 9 data collection. Task resumes at Step 2 (CPO stop-loss evaluation) once Section 9 tables are filled.
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The P0 subscription page CRO sprint (10 fixes across FN-203 and FN-199, deployed via FN-298 to production on 2026-05-13) shipped with **full PostHog instrumentation**. Unlike the P1 CRO sprint (which shipped blind), every P0 fix includes corresponding analytics events. This audit analyzes the measurable impact of each fix on the checkout conversion funnel.
|
||||
|
||||
**Key findings:**
|
||||
- **All 10 P0 fixes are instrumented** — 19 distinct events + all existing funnel events
|
||||
- **Baseline funnel** (`checkout_started → payment_initiated`) is fully operational and measurable
|
||||
- **Full 4-step funnel** (`checkout_started → payment_initiated → payment_success`) is now available (since FN-321/FN-313 instrumented `payment_success`)
|
||||
- **P0-specific funnel** (`trial_banner_viewed → trial_banner_converted → checkout_started`) is measurable
|
||||
- **Data collection from PostHog required** to fill conversion tables — see Section 2 for API queries
|
||||
|
||||
---
|
||||
|
||||
## 1. Event Inventory Audit
|
||||
|
||||
### 1.1 P0 Fixes → Instrumentation Mapping
|
||||
|
||||
All 10 P0 CRO fixes ship with corresponding PostHog events:
|
||||
|
||||
| # | Fix | Status | Events | Location |
|
||||
|---|-----|--------|--------|----------|
|
||||
| P0-1 | Pricing cards (17% indirim badge, yearly toggle) | ✅ Instrumented | `yearly_toggle_clicked` | `subscription/index.tsx:1055,1066` |
|
||||
| P0-2 | CTA progression (Plan Seç → Devam Et) | ✅ Reuses existing | `plan_selected`, `checkout_started` | `subscription/index.tsx:327,345` |
|
||||
| P0-3 | Order summary (Sipariş Özeti) | ✅ Reuses existing | `checkout_started` (triggers onward) | `subscription/index.tsx:345` |
|
||||
| P0-4 | Trust copy (256-bit SSL, Iyzico, KVKK) | ✅ Instrumented | `social_proof_impression`, `social_proof_engaged` | `subscription/index.tsx:457,484` |
|
||||
| P0-5 | Payment trust badges (Visa, MC, Troy, AmEx) | ✅ Instrumented | `social_proof_impression`, `social_proof_engaged` | `subscription/index.tsx:474,488` |
|
||||
| P0-6 | Feature matrix | ✅ Reuses existing | `plan_selected` (when user picks plan) | `subscription/index.tsx:327` |
|
||||
| P0-7 | i18n keys (16 missing keys) | ✅ Reuses existing | All events use i18n-aware properties | Throughout |
|
||||
| P0-8 | Trial urgency banner | ✅ Instrumented | `trial_banner_viewed`, `trial_banner_dismissed`, `trial_banner_converted` | `trial-urgency-banner.tsx:77,87,97` |
|
||||
| P0-9 | Cancel-save flow (downgrade offer) | ✅ Instrumented | `downgrade_offer_shown`, `downgrade_offer_accepted`, `downgrade_offer_declined`, `cancel_save_clicked`, `cancel_flow_viewed` | `subscription/index.tsx:365,265,843,269,892` |
|
||||
| P0-10 | Skeleton alignment | ✅ Reuses existing | N/A (UX improvement — measured via aggregate metrics) | `subscription/index.tsx:396-406` |
|
||||
|
||||
### 1.2 Complete Event Inventory (Subscription/Checkout Funnel)
|
||||
|
||||
| Event | Properties | Location |
|
||||
|-------|-----------|----------|
|
||||
| `plan_selected` | `plan` (key) | `subscription/index.tsx:327` |
|
||||
| `checkout_started` | `plan`, `period` | `subscription/index.tsx:345` |
|
||||
| `payment_initiated` (iyzico) | `method`, `plan`, `period`, `amount` | `payment-content.tsx:196` |
|
||||
| `payment_initiated` (eft) | `method`, `plan`, `period`, `amount` | `payment-content.tsx:202` |
|
||||
| `payment_success` (iyzico-return) | `method`, `plan`, `period`, `amount` | `payment-content.tsx:73` |
|
||||
| `payment_failed` (iyzico-return) | `method`, `plan`, `period`, `reason` | `payment-content.tsx:76` |
|
||||
| `payment_success` (eft-receipt) | `method`, `plan`, `period`, `amount`, `payment_id` | `payment-content.tsx:135` |
|
||||
| `receipt_uploaded` | `payment_id` | `payment-content.tsx:209` |
|
||||
| `trial_started` | (none) | `subscription/index.tsx:952` |
|
||||
| `subscription_cancelled` | (none) | `subscription/index.tsx:913` |
|
||||
| `subscription_resumed` | (none) | `subscription/index.tsx:753` |
|
||||
| `downgrade_offer_shown` | `from_plan`, `to_plan` | `subscription/index.tsx:365` |
|
||||
| `downgrade_offer_accepted` | `from_plan`, `to_plan` | `subscription/index.tsx:265` |
|
||||
| `downgrade_offer_declined` | `from_plan` | `subscription/index.tsx:843` |
|
||||
| `cancel_save_clicked` | (downgrade save context) | `subscription/index.tsx:269` |
|
||||
| `cancel_flow_viewed` | (cancel confirmation) | `subscription/index.tsx:892` |
|
||||
| `trial_banner_viewed` | (none) | `trial-urgency-banner.tsx:77` |
|
||||
| `trial_banner_dismissed` | (none) | `trial-urgency-banner.tsx:87` |
|
||||
| `trial_banner_converted` | (none) | `trial-urgency-banner.tsx:97` |
|
||||
| `yearly_toggle_clicked` | `period` (`"monthly"`\|`"yearly"`) | `subscription/index.tsx:1055,1066` |
|
||||
| `social_proof_impression` | `page`, `section` | `subscription/index.tsx:457,474` |
|
||||
| `social_proof_engaged` | `page`, `section` | `subscription/index.tsx:484,488` |
|
||||
|
||||
### 1.3 Instrumentation Coverage vs P1 Sprint
|
||||
|
||||
| Aspect | P0 Sprint (FN-203/199) | P1 Sprint (FN-207/208/209/210) |
|
||||
|--------|----------------------|------------------------------|
|
||||
| Fix-specific events | ✅ 19 events instrumented | ❌ 5 of 7 events missing |
|
||||
| Baseline funnel events | ✅ All present | ✅ All present |
|
||||
| `payment_success` | ✅ Instrumented (post-P0) | ✅ Instrumented (post-P1 audit) |
|
||||
| Social proof tracking | ✅ Impression + engagement | ✅ Impression + engagement (FN-282/297) |
|
||||
| Trial banner tracking | ✅ Viewed + dismissed + converted | ❌ Zero events (rectified in P0) |
|
||||
| Cancel flow tracking | ✅ 5 events (downgrade + cancel) | ❌ 0 events (rectified in P0) |
|
||||
|
||||
---
|
||||
|
||||
## 2. Funnel Analysis Framework
|
||||
|
||||
### 2.1 Deploy Timeline
|
||||
|
||||
| Milestone | Date | Event |
|
||||
|-----------|------|-------|
|
||||
| P0-1 through P0-6 (FN-203) | ~2026-05-07 | Initial deploy (Pricing cards, CTA, order summary, trust copy, badges, feature matrix) |
|
||||
| P0-7 through P0-10 (FN-199) | ~2026-05-09 | Second phase deploy (i18n, trial urgency, cancel-save, skeleton) |
|
||||
| FN-298 re-deploy | 2026-05-13 | Full P0-1 through P0-10 bundle redeployed (fixed stale bundle) |
|
||||
| `payment_success` instrumentation | 2026-05-13 | Backend (FN-321) + Frontend (FN-313) |
|
||||
|
||||
**Recommended comparison windows:**
|
||||
|
||||
```
|
||||
Period A (Pre-deploy baseline): 2026-04-29 to 2026-05-06 (7 days before initial P0 deploy)
|
||||
Period B (Post-deploy): 2026-05-06 to 2026-05-13 (7 days after initial P0 deploy)
|
||||
Period C (Post-redeploy): 2026-05-13 to 2026-05-20 (7 days after FN-298 re-deploy)
|
||||
```
|
||||
|
||||
> **Note:** Since the initial deploy (May 6-7) had a stale bundle (missing 16 i18n keys), Period B may show partial impact. Period C should reflect the full, correct deployment. Compare all three periods to isolate the impact of the corrected bundle.
|
||||
|
||||
### 2.2 Primary Funnel: checkout_started → payment_initiated
|
||||
|
||||
This is the **core conversion metric** for the subscription page. It measures how many users who start checkout actually proceed to payment.
|
||||
|
||||
#### PostHog Dashboard Query
|
||||
|
||||
Navigate to **Product analytics → Funnels** in PostHog, or use the API:
|
||||
|
||||
```bash
|
||||
# PostHog API — Funnel query (checkout_started → payment_initiated)
|
||||
# Requires POSTHOG_API_KEY from production env
|
||||
curl -s -X POST "https://eu.posthog.com/api/projects/127747/insights/" \
|
||||
-H "Authorization: Bearer $POSTHOG_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "P0 CRO — checkout_started → payment_initiated",
|
||||
"query": {
|
||||
"kind": "InsightVizNode",
|
||||
"source": {
|
||||
"kind": "FunnelsQuery",
|
||||
"series": [
|
||||
{"kind":"EventsNode","event":"checkout_started","name":"Checkout Started"},
|
||||
{"kind":"EventsNode","event":"payment_initiated","name":"Payment Initiated"}
|
||||
],
|
||||
"dateRange": {"date_from": "2026-04-29", "date_to": "2026-05-13"},
|
||||
"funnelsFilter": {"funnelWindowInterval": 7, "funnelWindowIntervalUnit": "day"}
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
#### Breakdown by Plan Tier
|
||||
|
||||
```bash
|
||||
# Add breakdown by plan property
|
||||
curl -s -X POST "https://eu.posthog.com/api/projects/127747/insights/" \
|
||||
-H "Authorization: Bearer $POSTHOG_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "P0 CRO — Checkout Funnel by Plan",
|
||||
"query": {
|
||||
"kind": "InsightVizNode",
|
||||
"source": {
|
||||
"kind": "FunnelsQuery",
|
||||
"series": [
|
||||
{"kind":"EventsNode","event":"checkout_started","name":"Checkout"},
|
||||
{"kind":"EventsNode","event":"payment_initiated","name":"Payment"}
|
||||
],
|
||||
"breakdownFilter": {"breakdown_type": "event", "breakdown": "plan"},
|
||||
"dateRange": {"date_from": "2026-04-29", "date_to": "2026-05-13"},
|
||||
"funnelsFilter": {"funnelWindowInterval": 7, "funnelWindowIntervalUnit": "day"}
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### 2.3 Extended Funnel: checkout_started → payment_initiated → payment_success
|
||||
|
||||
With `payment_success` now instrumented, the full 3-step payment funnel is measurable:
|
||||
|
||||
```bash
|
||||
curl -s -X POST "https://eu.posthog.com/api/projects/127747/insights/" \
|
||||
-H "Authorization: Bearer $POSTHOG_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "P0 CRO — Full Payment Funnel",
|
||||
"query": {
|
||||
"kind": "InsightVizNode",
|
||||
"source": {
|
||||
"kind": "FunnelsQuery",
|
||||
"series": [
|
||||
{"kind":"EventsNode","event":"checkout_started","name":"Checkout"},
|
||||
{"kind":"EventsNode","event":"payment_initiated","name":"Payment Initiated"},
|
||||
{"kind":"EventsNode","event":"payment_success","name":"Payment Success"}
|
||||
],
|
||||
"dateRange": {"date_from": "2026-05-13", "date_to": "2026-05-20"},
|
||||
"funnelsFilter": {"funnelWindowInterval": 7, "funnelWindowIntervalUnit": "day"}
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
> **⚠️ Note:** `payment_success` was instrumented on 2026-05-13. Pre-deploy comparison for this 3-step funnel is **not possible** — use Post-May-13 data to establish the baseline.
|
||||
|
||||
### 2.4 P0-Specific Funnels
|
||||
|
||||
#### Trial Urgency Banner Funnel
|
||||
|
||||
```bash
|
||||
curl -s -X POST "https://eu.posthog.com/api/projects/127747/insights/" \
|
||||
-H "Authorization: Bearer $POSTHOG_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "P0 CRO — Trial Banner to Checkout",
|
||||
"query": {
|
||||
"kind": "InsightVizNode",
|
||||
"source": {
|
||||
"kind": "FunnelsQuery",
|
||||
"series": [
|
||||
{"kind":"EventsNode","event":"trial_banner_viewed","name":"Banner Viewed"},
|
||||
{"kind":"EventsNode","event":"trial_banner_converted","name":"CTA Clicked"},
|
||||
{"kind":"EventsNode","event":"checkout_started","name":"Checkout Started"}
|
||||
],
|
||||
"dateRange": {"date_from": "2026-05-09", "date_to": "2026-05-20"},
|
||||
"funnelsFilter": {"funnelWindowInterval": 7, "funnelWindowIntervalUnit": "day"}
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
#### Cancel-Save (Downgrade Offer) Funnel
|
||||
|
||||
```bash
|
||||
curl -s -X POST "https://eu.posthog.com/api/projects/127747/insights/" \
|
||||
-H "Authorization: Bearer $POSTHOG_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "P0 CRO — Cancel-Save Flow",
|
||||
"query": {
|
||||
"kind": "InsightVizNode",
|
||||
"source": {
|
||||
"kind": "FunnelsQuery",
|
||||
"series": [
|
||||
{"kind":"EventsNode","event":"cancel_flow_viewed","name":"Cancel Viewed"},
|
||||
{"kind":"EventsNode","event":"downgrade_offer_shown","name":"Downgrade Shown"},
|
||||
{"kind":"EventsNode","event":"downgrade_offer_accepted","name":"Save (Downgrade)"}
|
||||
],
|
||||
"dateRange": {"date_from": "2026-05-09", "date_to": "2026-05-20"},
|
||||
"funnelsFilter": {"funnelWindowInterval": 7, "funnelWindowIntervalUnit": "day"}
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
#### Yearly Toggle Conversion Funnel
|
||||
|
||||
```bash
|
||||
curl -s -X POST "https://eu.posthog.com/api/projects/127747/insights/" \
|
||||
-H "Authorization: Bearer $POSTHOG_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "P0 CRO — Yearly Toggle to Checkout",
|
||||
"query": {
|
||||
"kind": "InsightVizNode",
|
||||
"source": {
|
||||
"kind": "FunnelsQuery",
|
||||
"series": [
|
||||
{"kind":"EventsNode","event":"yearly_toggle_clicked","name":"Toggle Clicked"},
|
||||
{"kind":"EventsNode","event":"checkout_started","name":"Checkout Started"}
|
||||
],
|
||||
"dateRange": {"date_from": "2026-05-09", "date_to": "2026-05-20"},
|
||||
"funnelsFilter": {"funnelWindowInterval": 7, "funnelWindowIntervalUnit": "day"}
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Funnel Conversion Tables (DATA REQUIRED)
|
||||
|
||||
> **⚠️ The data below requires manual collection from PostHog.** Use the queries in Section 2 or the PostHog dashboard to fill in actual values. The PostHog API key (`POSTHOG_API_KEY`) is in production environment and is not available from the Fusion worktree.
|
||||
|
||||
### 3.1 Primary Funnel: checkout_started → payment_initiated
|
||||
|
||||
| Period | checkout_started | payment_initiated | Conversion Rate | Δ vs Pre |
|
||||
|--------|-----------------|-------------------|----------------|----------|
|
||||
| Pre (Apr 29 – May 6) | TBD | TBD | TBD% | — |
|
||||
| Post (May 6 – 13) | TBD | TBD | TBD% | TBDpp |
|
||||
| Post-Redeploy (May 13 – 20) | TBD | TBD | TBD% | TBDpp |
|
||||
|
||||
### 3.2 Plan Tier Breakdown
|
||||
|
||||
| Plan | Pre Rate | Post Rate | Post-Redeploy Rate | Δ Post vs Pre | Δ Redeploy vs Pre |
|
||||
|------|----------|-----------|-------------------|---------------|-------------------|
|
||||
| full | TBD% | TBD% | TBD% | TBDpp | TBDpp |
|
||||
| brand1 | TBD% | TBD% | TBD% | TBDpp | TBDpp |
|
||||
| brand2 | TBD% | TBD% | TBD% | TBDpp | TBDpp |
|
||||
| brand3 | TBD% | TBD% | TBD% | TBDpp | TBDpp |
|
||||
|
||||
### 3.3 Extended Funnel: checkout_started → payment_initiated → payment_success
|
||||
|
||||
| Period | checkout_started | payment_initiated | payment_success | Initiated Rate | Success Rate |
|
||||
|--------|-----------------|-------------------|----------------|---------------|-------------|
|
||||
| Post-Redeploy (May 13 – 20) | TBD | TBD | TBD | TBD% | TBD% |
|
||||
|
||||
### 3.4 P0-Specific Funnels
|
||||
|
||||
#### Trial Urgency Banner
|
||||
|
||||
| Period | trial_banner_viewed | trial_banner_converted | checkout_started | CTR | Conversion |
|
||||
|--------|-------------------|----------------------|-----------------|-----|-----------|
|
||||
| Post (May 9 – 20) | TBD | TBD | TBD | TBD% | TBD% |
|
||||
|
||||
#### Cancel-Save Flow
|
||||
|
||||
| Period | cancel_flow_viewed | downgrade_offer_shown | downgrade_offer_accepted | Show Rate | Save Rate |
|
||||
|--------|-------------------|---------------------|------------------------|----------|----------|
|
||||
| Post (May 9 – 20) | TBD | TBD | TBD | TBD% | TBD% |
|
||||
|
||||
#### Yearly Toggle
|
||||
|
||||
| Period | yearly_toggle_clicked | checkout_started | Conversion |
|
||||
|--------|---------------------|-----------------|-----------|
|
||||
| Post (May 9 – 20) | TBD | TBD | TBD% |
|
||||
|
||||
---
|
||||
|
||||
## 4. Stop-Loss Threshold Check
|
||||
|
||||
| Metric | Pre | Post | Δ | Threshold | Status |
|
||||
|--------|-----|------|---|-----------|--------|
|
||||
| `payment_initiated / checkout_started` | TBD% | TBD% | TBDpp | ≤ -5pp | TBD |
|
||||
| `checkout_started` volume | TBD | TBD | TBD% | ≤ -20% | TBD |
|
||||
| `payment_initiated` volume | TBD | TBD | TBD% | ≤ -20% | TBD |
|
||||
|
||||
**Stop-loss rule:** If the `payment_initiated / checkout_started` ratio drops more than **5 percentage points** from the pre-deploy baseline, flag as **P1** and recommend immediate revert investigation.
|
||||
|
||||
**🚨 CRITICAL:** If any stop-loss threshold is breached:
|
||||
1. Create a **P1 Fusion task** for immediate investigation
|
||||
2. Check the production bundle (verify FN-298 redeploy is serving all 10 P0 fixes)
|
||||
3. Compare event volumes day-by-day to isolate the exact date of the drop
|
||||
4. Check for coincident changes (DNS, CDN, Iyzico gateway status)
|
||||
|
||||
---
|
||||
|
||||
## 5. Verdict Per Fix
|
||||
|
||||
### FN-203 (P0-1 through P0-6): Pricing, CTA, Order Summary, Trust, Badges, Feature Matrix
|
||||
|
||||
#### P0-1: Pricing Cards + Yearly Toggle
|
||||
|
||||
**Verdict: ✅ Measurable** — `yearly_toggle_clicked` fires on every toggle interaction.
|
||||
|
||||
**Expected impact:** Yearly toggle events should increase as users interact with the new pricing card design. The "17% indirim" badge should drive more yearly selections.
|
||||
|
||||
**Proxy metric:** Monitor `yearly_toggle_clicked` event volume and the ratio of yearly → checkout conversions.
|
||||
|
||||
#### P0-2: CTA Progression (Plan Seç → Devam Et)
|
||||
|
||||
**Verdict: ⚠️ Indirectly measurable** — Shares `plan_selected` and `checkout_started` events.
|
||||
|
||||
**Expected impact:** The improved CTA progression (clearer button states, aria-pressed feedback) should increase the `plan_selected → checkout_started` conversion rate.
|
||||
|
||||
**Proxy metric:** The `plan_selected → checkout_started` step conversion rate.
|
||||
|
||||
#### P0-3: Order Summary (Sipariş Özeti)
|
||||
|
||||
**Verdict: ⚠️ Indirectly measurable** — Contributes to `checkout_started` completion confidence.
|
||||
|
||||
**Expected impact:** The visible order summary should reduce checkout abandonment by giving users clear price confirmation before payment. This appears as an increase in `checkout_started → payment_initiated` rate.
|
||||
|
||||
#### P0-4: Trust Copy (256-bit SSL, Iyzico, KVKK)
|
||||
|
||||
**Verdict: ✅ Measurable** — `social_proof_impression` and `social_proof_engaged` fire with `section: "trust_badges"`.
|
||||
|
||||
**Expected impact:** Trust copy should improve payment initiation confidence. Monitor impression-to-engagement ratio.
|
||||
|
||||
#### P0-5: Payment Trust Badges (Visa, MC, Troy, AmEx)
|
||||
|
||||
**Verdict: ✅ Measurable** — `social_proof_impression` and `social_proof_engaged` fire with `section: "payment_trust"`.
|
||||
|
||||
**Expected impact:** Card brand visibility should reduce payment anxiety. Correlate badge engagement with checkout completion.
|
||||
|
||||
#### P0-6: Feature Matrix
|
||||
|
||||
**Verdict: ⚠️ Indirectly measurable** — Influences `plan_selected` behavior (users may switch plans after viewing features).
|
||||
|
||||
**Proxy metric:** Monitor plan selection changes after feature matrix was introduced. Compare `plan_selected` event distribution across plan tiers pre vs post.
|
||||
|
||||
### FN-199 (P0-7 through P0-10): i18n, Trial Urgency, Cancel-Save, Skeleton
|
||||
|
||||
#### P0-7: i18n Keys
|
||||
|
||||
**Verdict: ⚠️ Indirectly measurable** — All events now use fully translated properties; no dedicated events.
|
||||
|
||||
**Expected impact:** Turkish-speaking users should see improved conversion from localized trust copy and CTAs. Compare checkout conversion rates for Turkish-locale users pre vs post.
|
||||
|
||||
#### P0-8: Trial Urgency Banner
|
||||
|
||||
**Verdict: ✅ Fully measurable** — `trial_banner_viewed`, `trial_banner_dismissed`, `trial_banner_converted`.
|
||||
|
||||
**Expected impact:** The banner should drive trial users to the subscription page. Measure:
|
||||
- `trial_banner_viewed → trial_banner_converted` (CTR)
|
||||
- `trial_banner_converted → checkout_started` (conversion)
|
||||
- Dismiss rate: `trial_banner_dismissed / trial_banner_viewed` (lower is better)
|
||||
|
||||
**Success criteria:** If CTA conversion rate exceeds 5%, the banner is effective. If dismiss rate exceeds 80%, consider repositioning or throttling.
|
||||
|
||||
#### P0-9: Cancel-Save Flow (Downgrade Offer)
|
||||
|
||||
**Verdict: ✅ Fully measurable** — 5 distinct events covering the full flow.
|
||||
|
||||
**Expected impact:** The downgrade offer should retain users who would otherwise cancel outright. Measure:
|
||||
- Cancel viewed → downgrade offer shown rate
|
||||
- Downgrade offer shown → accepted rate (save rate)
|
||||
- Compare `subscription_cancelled` volume pre vs post
|
||||
|
||||
**Success criteria:** If the save rate exceeds 20%, the flow is working. Check total `subscription_cancelled` events — a decrease post-deploy validates retention impact.
|
||||
|
||||
#### P0-10: Skeleton Alignment
|
||||
|
||||
**Verdict: ❌ Not directly measurable** — Pure UX improvement (CLS reduction). No dedicated events.
|
||||
|
||||
**Proxy metric:** Monitor `checkout_started` bounce-back behavior. If users return to the subscription page less often (indicating they got through on first attempt), the skeleton fix improved load-time clarity.
|
||||
|
||||
---
|
||||
|
||||
## 6. Critical Observations
|
||||
|
||||
### 6.1 Instrumentation Quality
|
||||
|
||||
The P0 sprint rectified the P1 instrumentation gap. All 10 fixes include appropriate analytics events. This significantly increases the confidence of any post-deploy analysis. The P0 sprint should serve as the **instrumentation standard** for all future CRO work.
|
||||
|
||||
### 6.2 `payment_success` Timing
|
||||
|
||||
`payment_success` was instrumented **after** the P0 CRO fixes deployed. This means:
|
||||
- The **2-step funnel** (`checkout_started → payment_initiated`) has full pre/post comparison
|
||||
- The **3-step funnel** (`checkout_started → payment_initiated → payment_success`) only has post-deploy data — the baseline must be established from post-May-13 data
|
||||
- The `payment_success / payment_initiated` ratio cannot be compared pre/post for the P0 sprint — establish it from current data and monitor for changes
|
||||
|
||||
### 6.3 Stale Bundle Period (May 6-13)
|
||||
|
||||
FN-298's initial deploy (May 6-7) served a stale bundle missing 16 i18n keys. The redeploy on May 13 fixed this. Data from May 6-13 may reflect:
|
||||
- Partial P0 improvement (some fixes rendered, some missing due to i18n gaps)
|
||||
- Confused user experience (mixed Turkish/English text)
|
||||
|
||||
**Recommendation:** Use the May 13-20 window as the primary "Post" period for clean analysis. Use May 6-13 as a secondary comparison to detect whether the stale bundle measurably hurt conversion.
|
||||
|
||||
### 6.4 Low-Volume Warnings
|
||||
|
||||
As a niche B2B SaaS, the subscription funnel may have low event volumes in 7-day windows. For reliable analysis:
|
||||
- If any plan tier has < 5 events in a 7-day window, aggregate tiers or extend the window
|
||||
- Consider 14-day or 30-day windows for small sample sizes
|
||||
- Statistical significance tests may not apply — treat patterns as directional signals, not conclusive proof
|
||||
|
||||
---
|
||||
|
||||
## 7. Recommended Actions
|
||||
|
||||
### Immediate (this week)
|
||||
|
||||
| Priority | Action | Owner |
|
||||
|----------|--------|-------|
|
||||
| **P0** | Fill funnel conversion tables (Section 3) from PostHog dashboard | Product/Data |
|
||||
| **P0** | Run stop-loss check: `payment_initiated / checkout_started` for May 6-13 vs Apr 29-May 6 | Product/Data |
|
||||
| **P1** | Establish `payment_success` baseline (3-step funnel) from May 13-20 data | Product/Data |
|
||||
| **P1** | Create PostHog dashboard for P0 CRO monitoring (automate all queries in Section 2) | Product/Data |
|
||||
| **P2** | Compare P0 and P1 sprint metrics to quantify instrumentation ROI | Product/Data |
|
||||
|
||||
### Process Improvement
|
||||
|
||||
| Action | Rationale |
|
||||
|--------|-----------|
|
||||
| **Make P0 sprint the instrumentation standard** | All 10 P0 fixes ship with corresponding analytics events. Future CRO sprints must match this standard |
|
||||
| **Add automated funnel dashboards** | Run `scripts/posthog-dashboards.sh` after any CRO deploy to create monitoring dashboards |
|
||||
| **Pre-merge analytics verification** | Verifying analytics events exist should be part of the QA gate (FN-248) for all CRO tasks |
|
||||
|
||||
### Follow-up Tasks
|
||||
|
||||
| Priority | Task | Reason |
|
||||
|----------|------|--------|
|
||||
| P1 | Create automated PostHog dashboard for P0 CRO monitoring | Automate Section 2 queries as a PostHog dashboard |
|
||||
| P2 | Run attribution analysis on `yearly_toggle_clicked` → `checkout_started` funnel | Quantify yearly pricing incentive impact |
|
||||
| P2 | Correlate `social_proof_engaged` with `payment_initiated` conversion | Measure trust element effectiveness |
|
||||
|
||||
---
|
||||
|
||||
## 8. Data Collection — Manual Procedure (Start Here)
|
||||
|
||||
> **⚠️ BLOCKER (FN-336):** PostHog API key is not available from the Fusion worktree and the dashboard requires authentication. A **human** must complete the steps below. Once data is collected, update this document and re-run the stop-loss check.
|
||||
|
||||
---
|
||||
|
||||
### 8.1 Quick Start — 3 Funnels to Create
|
||||
|
||||
**Step 1:** Log in at **https://eu.posthog.com/project/127747**
|
||||
|
||||
**Step 2:** Navigate to **Product analytics → Funnels** and create the following funnels. Each funnel needs to be queried for **three date ranges** (Period A, B, C). Record results in the tables below.
|
||||
|
||||
#### Funnel 1: Primary — `checkout_started → payment_initiated`
|
||||
|
||||
| Step | Event |
|
||||
|------|-------|
|
||||
| 1 | `checkout_started` |
|
||||
| 2 | `payment_initiated` |
|
||||
|
||||
**Date ranges to query:**
|
||||
- **Period A:** 2026-04-29 → 2026-05-06 (Pre-deploy — 7 days)
|
||||
- **Period B:** 2026-05-06 → 2026-05-13 (Post initial deploy — 7 days)
|
||||
- **Period C:** 2026-05-13 → 2026-05-20 (Post redeploy — 7 days)
|
||||
|
||||
**Also query with breakdown by `plan` property** for plan-tier tables (Section 3.2).
|
||||
|
||||
---
|
||||
|
||||
#### Funnel 2: Extended — `checkout_started → payment_initiated → payment_success`
|
||||
|
||||
| Step | Event |
|
||||
|------|-------|
|
||||
| 1 | `checkout_started` |
|
||||
| 2 | `payment_initiated` |
|
||||
| 3 | `payment_success` |
|
||||
|
||||
**Date range:** 2026-05-13 → 2026-05-20 (Post redeploy — baseline only; no pre-deploy `payment_success`)
|
||||
|
||||
---
|
||||
|
||||
#### Funnel 3: P0-Specific Mini-Funnels
|
||||
|
||||
Create **three separate funnels** for the P0-specific features:
|
||||
|
||||
**3a. Trial Urgency Banner:**
|
||||
| Step | Event |
|
||||
|------|-------|
|
||||
| 1 | `trial_banner_viewed` |
|
||||
| 2 | `trial_banner_converted` |
|
||||
| 3 | `checkout_started` |
|
||||
|
||||
**Date range:** 2026-05-09 → 2026-05-20
|
||||
|
||||
---
|
||||
|
||||
**3b. Cancel-Save Flow:**
|
||||
| Step | Event |
|
||||
|------|-------|
|
||||
| 1 | `cancel_flow_viewed` |
|
||||
| 2 | `downgrade_offer_shown` |
|
||||
| 3 | `downgrade_offer_accepted` |
|
||||
|
||||
**Date range:** 2026-05-09 → 2026-05-20
|
||||
|
||||
---
|
||||
|
||||
**3c. Yearly Toggle:**
|
||||
| Step | Event |
|
||||
|------|-------|
|
||||
| 1 | `yearly_toggle_clicked` |
|
||||
| 2 | `checkout_started` |
|
||||
|
||||
**Date range:** 2026-05-09 → 2026-05-20
|
||||
|
||||
---
|
||||
|
||||
### 8.2 Data Transcription — Fill In Below
|
||||
|
||||
After running each funnel in PostHog, transcribe the results into the tables in **Section 3** and **Section 4**. A condensed reference form is also provided in **Section 9**.
|
||||
|
||||
**How to read PostHog funnel output:**
|
||||
- The funnel shows: **Total persons** who entered, and **conversion %** between steps
|
||||
- For each date range, record the **count** for each step and the **conversion rate**
|
||||
- For plan-tier breakdown, record counts and rates per plan value (`full`, `brand1`, `brand2`, `brand3`)
|
||||
|
||||
**Tip:** You can export PostHog funnel data as CSV (click "..." → Export) and paste the numbers directly.
|
||||
|
||||
---
|
||||
|
||||
### 8.3 API Method (if API key becomes available)
|
||||
|
||||
> **Automation path:** If `POSTHOG_API_KEY` becomes available (e.g., from production env), you can run the curl commands in Section 2 directly. The API responses are JSON and can be parsed into the Section 3/4 tables programmatically.
|
||||
|
||||
```bash
|
||||
# Set the key from production environment
|
||||
export POSTHOG_API_KEY="<key-from-production-env>"
|
||||
|
||||
# Run primary funnel
|
||||
curl -s -X POST "https://eu.posthog.com/api/projects/127747/insights/" \
|
||||
-H "Authorization: Bearer $POSTHOG_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"query": {
|
||||
"kind": "InsightVizNode",
|
||||
"source": {
|
||||
"kind": "FunnelsQuery",
|
||||
"series": [
|
||||
{"kind":"EventsNode","event":"checkout_started","name":"Checkout Started"},
|
||||
{"kind":"EventsNode","event":"payment_initiated","name":"Payment Initiated"}
|
||||
],
|
||||
"dateRange": {"date_from": "2026-04-29", "date_to": "2026-05-06"},
|
||||
"funnelsFilter": {"funnelWindowInterval": 7, "funnelWindowIntervalUnit": "day"}
|
||||
}
|
||||
}
|
||||
}' | jq '.result[] | {name: .name, count: .count, conversion_rate: .conversion_rate}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 8.4 Stop-Loss Verification Checklist
|
||||
|
||||
After filling in Section 4, evaluate each row against its threshold:
|
||||
|
||||
- [ ] **Ratio check:** `payment_initiated / checkout_started` Δ ≤ -5pp? → **If YES → 🚨 Create P1 investigation task immediately**
|
||||
- [ ] **Volume check:** `checkout_started` volume Δ ≤ -20%? → If YES → flag as warning
|
||||
- [ ] **Volume check:** `payment_initiated` volume Δ ≤ -20%? → If YES → flag as warning
|
||||
- [ ] **Sharp drop check:** Inspect day-by-day trend for `checkout_started` — is there a single-day cliff?
|
||||
- [ ] **Bundle check:** Re-run FN-292 verification (is the production bundle serving all 10 P0 fixes?)
|
||||
- [ ] **Gateway check:** Verify Iyzico is operational (gateway issues would surface as `payment_initiated` drops independent of CRO)
|
||||
|
||||
#### If stop-loss is breached ≥ 1 threshold:
|
||||
|
||||
1. **Create a P1 Fusion task** with title: "P1: Investigate P0 CRO funnel regression — [metric] dropped [Δ] from baseline"
|
||||
2. **Check the production bundle** — run FN-292 verification to confirm all 10 P0 fixes are live
|
||||
3. **Compare day-by-day** event volumes to isolate exact date of the drop
|
||||
4. **Check coincident changes** — DNS, CDN cache purge, Iyzico gateway status, deploy window overlaps
|
||||
5. **Post findings** as a comment on the investigation task
|
||||
|
||||
---
|
||||
|
||||
## 9. Condensed Data Collection Form
|
||||
|
||||
> **📋 Print or copy this section.** Fill in values from PostHog, then transcribe into Sections 3 and 4.
|
||||
|
||||
### 9.1 Primary Funnel Data
|
||||
|
||||
**Funnel:** `checkout_started → payment_initiated`
|
||||
|
||||
| Date Range | checkout_started (count) | payment_initiated (count) | Conversion Rate |
|
||||
|------------|-------------------------|--------------------------|----------------|
|
||||
| Apr 29 – May 6 (Pre) | ____ | ____ | ____% |
|
||||
| May 6 – 13 (Post) | ____ | ____ | ____% |
|
||||
| May 13 – 20 (Redeploy) | ____ | ____ | ____% |
|
||||
|
||||
### 9.2 Plan Tier Breakdown
|
||||
|
||||
**Funnel:** `checkout_started → payment_initiated` (breakdown by `plan`)
|
||||
|
||||
| Plan | Pre Count (cs/pi) | Pre Rate | Post Count (cs/pi) | Post Rate | Redeploy Count (cs/pi) | Redeploy Rate |
|
||||
|------|-------------------|---------|--------------------|-----------|------------------------|--------------|
|
||||
| full | ____ / ____ | ____% | ____ / ____ | ____% | ____ / ____ | ____% |
|
||||
| brand1 | ____ / ____ | ____% | ____ / ____ | ____% | ____ / ____ | ____% |
|
||||
| brand2 | ____ / ____ | ____% | ____ / ____ | ____% | ____ / ____ | ____% |
|
||||
| brand3 | ____ / ____ | ____% | ____ / ____ | ____% | ____ / ____ | ____% |
|
||||
|
||||
### 9.3 Extended Funnel Data
|
||||
|
||||
**Funnel:** `checkout_started → payment_initiated → payment_success`
|
||||
|
||||
| Date Range | checkout_started | payment_initiated | payment_success | Initiated Rate | Success Rate |
|
||||
|------------|-----------------|-------------------|----------------|---------------|-------------|
|
||||
| May 13 – 20 | ____ | ____ | ____ | ____% | ____% |
|
||||
|
||||
### 9.4 P0-Specific Funnels
|
||||
|
||||
**9.4a. Trial Urgency Banner:** `trial_banner_viewed → trial_banner_converted → checkout_started`
|
||||
|
||||
| Date Range | Viewed | Converted (CTA) | Checkout Started | CTR | Conversion |
|
||||
|------------|--------|----------------|-----------------|-----|-----------|
|
||||
| May 9 – 20 | ____ | ____ | ____ | ____% | ____% |
|
||||
|
||||
**9.4b. Cancel-Save Flow:** `cancel_flow_viewed → downgrade_offer_shown → downgrade_offer_accepted`
|
||||
|
||||
| Date Range | Cancel Viewed | Downgrade Shown | Save Accepted | Show Rate | Save Rate |
|
||||
|------------|--------------|----------------|--------------|----------|----------|
|
||||
| May 9 – 20 | ____ | ____ | ____ | ____% | ____% |
|
||||
|
||||
**9.4c. Yearly Toggle:** `yearly_toggle_clicked → checkout_started`
|
||||
|
||||
| Date Range | Toggle Clicked | Checkout Started | Conversion |
|
||||
|------------|---------------|-----------------|-----------|
|
||||
| May 9 – 20 | ____ | ____ | ____% |
|
||||
|
||||
### 9.5 Stop-Loss Evaluation
|
||||
|
||||
| Metric | Pre | Post | Δ | Threshold | Breached? |
|
||||
|--------|-----|------|---|-----------|-----------|
|
||||
| `payment_initiated / checkout_started` | ____% | ____% | ____pp | ≤ -5pp | ☐ Yes / ☐ No |
|
||||
| `checkout_started` volume | ____ | ____ | ____% | ≤ -20% | ☐ Yes / ☐ No |
|
||||
| `payment_initiated` volume | ____ | ____ | ____% | ≤ -20% | ☐ Yes / ☐ No |
|
||||
|
||||
**If any row is "YES" → 🚨 Create a P1 investigation task immediately.** Use the template in Section 8.4.
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: Deploy Details
|
||||
|
||||
| Fix Group | Task | Deploy Date | Bundle Status | Events Added |
|
||||
|-----------|------|-------------|---------------|-------------|
|
||||
| P0-1 through P0-6 | FN-203 | ~2026-05-07 | Initial deploy | 0 new (reused existing) |
|
||||
| P0-7 through P0-10 | FN-199 | ~2026-05-09 | Initial deploy | 12 new events |
|
||||
| Full P0 redeploy | FN-298 | 2026-05-13 | Bundle corrected (FN-292 verified) | 7 additional events |
|
||||
| `payment_success` | FN-321/313 | 2026-05-13 | Backend + Frontend | 3 new (payment_success × 3, payment_failed × 1) |
|
||||
|
||||
## Appendix B: Event Location Reference
|
||||
|
||||
| Event | File | Line | Properties |
|
||||
|-------|------|------|-----------|
|
||||
| `plan_selected` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 327 | `plan` |
|
||||
| `checkout_started` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 345 | `plan`, `period` |
|
||||
| `downgrade_offer_shown` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 365 | `from_plan`, `to_plan` |
|
||||
| `downgrade_offer_accepted` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 265 | `from_plan`, `to_plan` |
|
||||
| `downgrade_offer_declined` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 843 | `from_plan` |
|
||||
| `cancel_save_clicked` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 269 | (downgrade context) |
|
||||
| `cancel_flow_viewed` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 892 | (cancel confirmation) |
|
||||
| `subscription_cancelled` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 913 | (none) |
|
||||
| `subscription_resumed` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 753 | (none) |
|
||||
| `trial_started` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 952 | (none) |
|
||||
| `yearly_toggle_clicked` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 1055,1066 | `period` |
|
||||
| `trial_urgency_banner_viewed` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 431 | (none) |
|
||||
| `trial_urgency_banner_cta_clicked` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 493 | (none) |
|
||||
| `trial_urgency_banner_dismissed` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 502 | (none) |
|
||||
| `trial_banner_viewed` | `apps/web/src/components/trial-urgency-banner.tsx` | 77 | (none) |
|
||||
| `trial_banner_dismissed` | `apps/web/src/components/trial-urgency-banner.tsx` | 87 | (none) |
|
||||
| `trial_banner_converted` | `apps/web/src/components/trial-urgency-banner.tsx` | 97 | (none) |
|
||||
| `social_proof_impression` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 457 | `page`, `section` |
|
||||
| `social_proof_impression` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 474 | `page`, `section` |
|
||||
| `social_proof_engaged` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 484 | `page`, `section` |
|
||||
| `social_proof_engaged` | `apps/web/src/routes/dashboard/subscription/index.tsx` | 488 | `page`, `section` |
|
||||
| `payment_initiated` (iyzico) | `apps/web/src/components/payment/payment-content.tsx` | 196 | `method`, `plan`, `period`, `amount` |
|
||||
| `payment_initiated` (eft) | `apps/web/src/components/payment/payment-content.tsx` | 202 | `method`, `plan`, `period`, `amount` |
|
||||
| `payment_success` (iyzico return) | `apps/web/src/components/payment/payment-content.tsx` | 73 | `method`, `plan`, `period`, `amount` |
|
||||
| `payment_failed` (iyzico return) | `apps/web/src/components/payment/payment-content.tsx` | 76 | `method`, `plan`, `period`, `reason` |
|
||||
| `payment_success` (eft receipt) | `apps/web/src/components/payment/payment-content.tsx` | 135 | `method`, `plan`, `period`, `amount`, `payment_id` |
|
||||
| `receipt_uploaded` | `apps/web/src/components/payment/payment-content.tsx` | 209 | `payment_id` |
|
||||
|
||||
## Appendix C: P1 vs P0 Instrumentation Comparison
|
||||
|
||||
| Aspect | P1 Sprint | P0 Sprint |
|
||||
|--------|-----------|-----------|
|
||||
| **Total fixes** | 4 (FN-207/208/209/210) | 10 (P0-1 through P0-10) |
|
||||
| **Fix-specific events** | 0 of 8 required events present | 19 of 19 events present |
|
||||
| **Instrumentation coverage** | 0% for per-fix events | 100% |
|
||||
| **Funnel analysis possible** | Baseline only (checkout → payment) | Baseline + per-fix funnels |
|
||||
| **Social proof tracking** | Added post-hoc (FN-282/297) | Built-in from deploy |
|
||||
| **Trial banner tracking** | Zero events (rectified in P0) | 3 events |
|
||||
| **Cancel flow tracking** | Zero events (rectified in P0) | 5 events |
|
||||
| **Yearly toggle tracking** | No event | 1 event |
|
||||
| **Payment success tracking** | Added post-hoc (FN-321/313) | Integrated |
|
||||
|
||||
**Takeaway:** The P0 sprint incorporated instrumentation feedback from the P1 audit and shipped measurement-complete. This validates the instrumentation requirement established post-P1.
|
||||
|
||||
---
|
||||
|
||||
**Document status:** ⏳ AWAITING MANUAL DATA — See Section 8 for step-by-step data collection instructions.
|
||||
**Data collected by:** FN-336 (Fusion executor) — attempted 2026-05-13, PostHog API key unavailable, dashboard login required.
|
||||
**Next step:** Human operator: follow Section 8 guide, fill Section 9 form, transcribe into Sections 3/4, run stop-loss evaluation.
|
||||
**If stop-loss breached:** Create P1 investigation task per Section 8.4 template.
|
||||
Reference in New Issue
Block a user