This podcast is all about digital experimentation. We discuss why it’s useful, why it’s harder that it seems, and best practices. Our guest is Dean Eckles, who is an assistant professor at MIT Sloan and previously worked at Facebook as a research scientist. We begin the conversation by describing how the human-computer interaction approach to studying digital systems differs from the marketing approach. We then discuss digital experiments, and how they can be used to both study policy and to learn about behavior. We then discuss when ‘big data’ is necessary for experiments to be informative and the possibility of non-parametric methods for inference. Next, we move on to Dean’s research on peer effects between users on Facebook. We finish with advice for researchers collaborating with the private sector.
References:
- Channelling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results by Alwyn Young
- Estimating peer effects in networks with peer encouragement designs by Eckles, D., Kizilcec, R.F. & Bakshy, E.
- Social influence in social advertising: Evidence from field experiments by Bakshy, E., Eckles, D., Yan, R., & Rosenn, I.
- Designing and deploying online field experiments by Bakshy, E., Eckles, D., Bernstein, M.
- Planout: A Framework for Online Field Experiments
- Experimental and Quasi-Experimental Analysis of Peer Effects: Two Steps Forward? by Bruce Sacerdote