Files and links (1)
SubmittedarXiv.org - Non-exclusive license to distribute, Open
Journal article
Latent Stratification for Incrementality Experiments
Marketing science (Providence, R.I.), v 43(4), pp 903-917
05 Jan 2024
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
A method is proposed to compute more precise standard errors for marketing A/B tests.
Incrementality experiments compare customers exposed to a marketing action designed to increase sales with those randomly assigned to a control group. These experiments suffer from noisy responses, which make precise estimation of the average treatment effect (ATE) and marketing return difficult. We develop a model that improves the precision by estimating separate treatment effects for three latent strata defined by potential outcomes in the experiment—customers who would buy regardless of ad exposure, those who would buy only if exposed to ads, and those who would not buy regardless. The overall ATE is estimated by averaging the strata-level effects, and this produces a more precise estimator of the ATE over a wide range of conditions typical of marketing experiments. Analytical results and simulations show that the method decreases the sampling variance of the ATE most when (1) there are large differences in the treatment effect between latent strata and (2) the model used to estimate the strata-level effects is well identified. Applying the procedure to five catalog experiments shows a reduction of 30%–60% in the variance of the overall ATE. This leads to a substantial decrease in decision errors when the estimator is used to determine whether ads should be continued or discontinued. History: Olivier Toubia served as the senior editor. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mksc.2022.0297 .
Metrics
30 Record Views
Details
- Title
- Latent Stratification for Incrementality Experiments
- Creators
- Ron Berman - University of PennsylvaniaElea McDonnell Feit - Drexel University
- Publication Details
- Marketing science (Providence, R.I.), v 43(4), pp 903-917
- Publisher
- Informs
- Number of pages
- 15
- Grant note
- Funding: This work was supported by Adobe Systems [Adobe Digital Experience Research Award]. R. Berman was supported by the United States-Israel Binational Science Foundation [Grant 2020022] and the Wharton Dean’s Research Fund, and E. M. Feit was supported by the Dean’s Fellowship from LeBow College of Business.
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Marketing
- Web of Science ID
- WOS:001141886700001
- Scopus ID
- 2-s2.0-85201050509
- Other Identifier
- 991021817944504721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Business