Synthetic Respondent Modeling

Synthetic respondent modeling uses a language model as a stand-in for human survey participants. It generates responses to hypothetical experimental conditions so researchers can estimate likely effects before recruiting a real sample.

The researcher describes a target population, experimental stimuli, and response scale in prompts, then samples many model-generated responses under treatment and control conditions. Comparing those response distributions produces an estimated treatment effect, much like comparing outcomes across arms of a conventional experiment. The method can also test studies that were not published before the model's training cutoff, although this does not prove that the model understands the underlying causal process.

The approach is faster and cheaper than recruiting large participant groups, and it can support pilot testing, intervention selection, and prioritization of replications without exposing people to potentially harmful stimuli. It may also help researchers explore many experimental designs before committing scarce funding or participant time. However, generated respondents inherit biases and defaults from the model, may fail to represent demographic and cultural variation, and cannot naturally reproduce non-response or social interaction. Large-scale evidence suggests that the method can track real experimental effects while systematically overstating their size.

It remains unclear whether successful forecasts come from genuine modeling of human decision-making, memorized associations, or a mixture of both. Researchers still need to establish how well the method transfers across countries, populations, topics, model families, and experiment types. Better calibration, transparency about prompting choices, and safeguards against replacing human participants prematurely are open research problems.