Double Jeopardy: Failed Project Gate, Inconclusive Replication
Table of Contents
Correction — August 2026
The original article treated r=0.627 as evidence that Double Jeopardy had failed in the beauty category and recommended moving budget from loyalty to reach. Those conclusions are withdrawn. The run failed an internal project gate, but construct-validity and bootstrap defects prevent a claim about the empirical law or a budget decision.
Read the corrected EBM-2025 v0.2 report or the v0.2 PDF. The original v0.1 PDF is archived and superseded.
Series navigation
- Duplication of Purchase analysis
- Category Entry Points analysis
- Buyer-frequency and NBD analysis
- Corrected analysis status
TL;DR
The archived run produced Pearson r=0.627, below the project’s 0.80 gate, and its stationarity check failed. The output therefore did not satisfy the project’s acceptance rule.
It does not follow that the Double Jeopardy law failed in a beauty market. The analyzed source was a giftware-retailer dataset transformed through heuristic category and label rules, and the reported bootstrap interval was generated by a procedure inconsistent with the point estimate. The result is an unsuccessful project test whose theoretical interpretation remains unresolved.
Background
Double Jeopardy is an empirical pattern in which lower-penetration brands tend to have fewer buyers and slightly lower purchase frequency among those buyers. A basic diagnostic plots brand penetration against average transaction frequency among buyers and measures their association across brands.
Three distinctions matter here:
- A correlation below a project threshold is not the same as evidence against the empirical law.
- A statistically non-zero correlation is not the same as meeting a pre-set strength criterion.
- A valid test requires stable market, category, brand, buyer, and time-window definitions before correlation is calculated.
Archived design
The pipeline used UCI Online Retail II invoice data. The source describes transactions from a UK giftware retailer. Project-specific processing created a bodycare category and derived brand-like labels from product text.
For each retained label, the script calculated:
- penetration: unique buyers of the label divided by all unique users in the processed slice;
- average frequency: transaction-row count divided by unique buyers of the label;
- Pearson and Spearman correlations across retained labels;
- weekly drift and Kendall trend checks;
- a reported 5,000-draw user bootstrap.
The selected output used a 26-week slice and retained 27 labels and 1,264 users. A later summary described 16 brands under a different threshold. Because those reported counts refer to different filtering states, they should not be combined into one sample description.
Archived result
The UCI run retained 27 derived labels and 1,264 users in a 26-week slice. Across those labels, buyer penetration and average transaction frequency among buyers had Pearson r=0.6269 and Spearman r=0.5624.
| Metric | Archived output | Gate or check | Decision |
|---|---|---|---|
| Pearson correlation | 0.627 | ≥0.80 | FAIL |
| Spearman correlation | 0.562 | descriptive | — |
| Pearson p-value | 0.00047 | not a fit gate | — |
| Logged stationarity | false | required for stable interpretation | FAIL |
| Maximum logged drift | 0.375 | ≤0.10 under the script rule | FAIL |
The result does not support the project’s strong Double Jeopardy criterion. It also does not establish a failed replication of the law. The UCI source describes a UK giftware retailer; the “bodycare” category and brand-like labels were created by project heuristics. The dataset therefore does not provide a secure beauty-market or brand construct.
Figure 1. Archived penetration-frequency scatter plot. Pearson r=0.627 was computed across the retained labels; it was not a top-decile statistic.
Stationarity result
The weekly diagnostic recorded a maximum drift of 0.375 against the script’s 0.10 rule and marked stationarity as false. This matters because penetration and frequency measured over a changing slice need not describe one stable market process.
The stationarity failure narrows the result; it does not reveal the cause. Changes in assortment, customer mix, label construction, seasonality, or the retailer’s transaction process could all contribute. The archived analysis did not identify among those mechanisms.
Withdrawn top-decile claim
The script contains no middle-quantile or top-decile Double Jeopardy split. The value r=0.627 was computed across all 27 retained labels. Claims that the relationship held in the middle and weakened among the highest-penetration 10% have no corresponding analysis in the archived code.
Why the interval is not used
The logged interval [0.275, 0.462] does not contain the point estimate 0.627. The audit identified two causes:
- penetration was computed with a denominator different from the point-estimate path;
- users sampled with replacement were reduced to set membership, so repeated draws received no additional weight.
The procedure also did not calculate the bias-correction and acceleration terms required for a BCa interval. The logged bounds are not interpretable as uncertainty around the reported point estimate.
Sensitivity claims from the original article
The original version listed correlations for alternative buyer thresholds and 52-week windows, then attributed their movement to inherent category structure. The code audit did not establish a complete, commit-linked output set for those combinations. Those values are not used here to distinguish category effects from methodological effects.
Even a fully reproduced sensitivity grid would not identify an “inherent” beauty-category mechanism unless the underlying category and brand mapping were first validated. Sensitivity to parameters can show that an estimate moves; it cannot by itself explain why.
Reproduction record
The archived article gave this command:
poetry run python scripts/eb/compute_dj.py \
--tx data/processed/tx_uci_beauty_with_categories.csv \
--category_regex bodycare \
--min_buyers 10 \
--window_weeks 26
The UCI log recorded its input only as loaded and its commit as unknown. The command documents the intended entry point but is insufficient to establish exact provenance.
Claim boundary
An internally defined gate failure is not evidence that loyalty programs are ineffective, that reach investment has higher returns, or that Double Jeopardy has limited applicability to a product category. Those claims require valid market and brand units plus a design that connects the descriptive law to the proposed decision.
Reimplementation requirements
A new test should begin with a source whose category and brands are directly observed or independently validated. The analysis should freeze its observation window and minimum-buyer rule, keep bootstrap multiplicity, use the same denominator in point and interval estimates, and record every attempted specification. A failed pre-set gate can then be reported as a failed test without converting it into a causal budget recommendation or a universal theory claim.
References
- Ehrenberg, A.S.C. (1988). Repeat-Buying: Facts, Theory and Applications.
- Sharp, B. (2010). How Brands Grow.
EBM-2025 v0.2 — Replication Audit
Read the corrected report, including the withdrawn claims, code-audit findings, methods, and limits.
Related Posts
- The API Hypothesis Was Not the Final Product
- Idea to Service OS v1: A Repeatable Method for AI-built Business Systems
- Reusable Asset Loop: Converting One Case into Repeatable Service Components
- Evidence-first Service Development: A Method to Prevent AI Product Overclaim
- Externalized Cognition for Solo Builders: Turning Notes, Prompts, and Checklists into a Build System
- Operations-first AI Service Design: Privacy, Review, and Launch Constraints
- GTM-attached Development: Designing PoC and LoI as Product Inputs