New paper in Q Open
Together with Julian Sagebiel and Jens Rommel, we published a paper entitled “A Pre-Registration Template for Transparent Choice Experiment Research” in Q Open, advocating for the use of pre-registration templates for choice experiments. The article can be found here.
This article grew out of a simple frustration that generic pre-registration templates (like the ones from the Open Science Framework) work fine for a randomized controlled trial, but they don’t really ask the questions that matter for a discrete choice experiment. Things like how you selected your bid vector or priors for an efficient design, whether you’re using a random utility or random regret model, or how you plan to handle preference heterogeneity and protest responses.
So we built a template that does. It’s organized into four parts: general information (hypotheses, sample, exclusion criteria), experimental design (choice scenario, statistical design, priors), estimation (welfare measures, heterogeneity, handling of outliers), and validity (follow-up questions, budget reminders, and the like). The idea isn’t to force researchers to commit to a specific model before they’ve even seen the data. That would be unrealistic, since things like distributional assumptions for random parameters often can’t be decided from theory alone. Instead, the template asks researchers to document their decision rules in advance, so it’s transparent which choices were fixed before data collection and which were made afterward.
We think this kind of transparency matters especially for our field, where welfare estimates often feed directly into policy decisions, and where critics have long argued that choice experiments can be (deliberately or not) designed or analyzed in ways that push results in a particular direction.
The template is open source and available on Zenodo in several formats (PDF, DOCX, HTML, LaTeX, and Quarto), and the source code lives on GitHub, where we’re hoping others in the community will help us improve it over time.