The paper investigates whether large language models (specifically GPT‑4o) can faithfully simulate individual U.S. adults by constructing generative agents from three data sources: semi‑structured AI‑conducted interviews, structured survey responses (General Social Survey and BFI‑44), and basic demographic information. A stratified sample of 1,052 participants completed two‑hour AI‑driven interviews, surveys, incentivized economic games, and five experimental tasks, with a two‑week retest wave. For each participant three agents were built (interview‑only, survey‑only, interview+survey) and compared against demographic‑only and persona‑description baselines. Performance was measured on held‑out GSS items, Big‑Five personality traits, and economic‑game outcomes, using normalized accuracy or correlation relative to each participant’s test‑retest consistency. Key findings: (1) interview‑, survey‑, and combined agents substantially outperformed demographic and persona baselines, recovering 82‑86% of the participants’ test‑retest benchmark on the GSS; (2) agents predicted personality traits and experimental effect directions; (3) interview‑based agents reduced accuracy disparities across political ideology, race, and sexual orientation; (4) a novel AI voice interviewer enabled scalable, high‑quality semi‑structured data collection; (5) robustness analyses showed that even with substantial removal of interview utterances performance degrades gracefully, and maximal agents (using all data sources) achieve the highest normalized accuracy (0.87) on the GSS. The paper also details the interview script, quota‑based sampling strategy, regression analyses of demographic influences on predictive performance, demographic parity difference (DPD) metrics across agent types, and discusses limitations such as reliance on a single model family, English‑only data, and under‑powered replication sets.
Topics
Study Design, Sampling, and AI‑Driven Interview Protocol
Details the stratified quota‑based sampling strategy used to match U.S. census demographics, the achieved representation across age, gender, region, race/ethnicity, education, and partisanship, and the AI voice interviewer with its full semi‑structured interview script (life history, health, finances, politics, social identity, values) that enabled scalable, high‑quality self‑report data collection for agent construction.
Agent Configurations and Construction
Describes the five agent types built for each participant—interview‑only, survey‑only (GSS + BFI‑44), interview + survey, demographic‑only, and persona‑description baselines—detailing the data sources (AI‑conducted interview, structured surveys, basic demographics) and the pipeline used to instantiate generative agents from these inputs.
Predictive Performance and Normalized Benchmarks
Presents the core results: agents’ accuracy on held‑out GSS items, Big‑Five personality traits, and economic‑game outcomes, using metrics such as raw accuracy, normalized accuracy (relative to participants’ test‑retest consistency), mean absolute error, and correlation. Highlights that interview, survey, and combined agents recover 82‑86 % of the test‑retest benchmark, with maximal agents reaching a normalized accuracy of 0.87 on the GSS.
Demographic Analyses and Fairness Evaluation
Combines regression analyses of demographic variables (age, region, ideology, party, education, race, ethnicity, gender, income, neighborhood, sexual orientation) on agent predictive performance with Demographic Parity Difference (DPD) metrics across agent configurations. Shows that interview‑based agents markedly reduce accuracy disparities across political ideology, race, and sexual orientation.
Robustness and Data Ablation Studies
Reports systematic robustness experiments that lesion portions of the interview transcript (random removal) and vary hold‑out strategies to assess performance degradation on GSS, Big‑Five, and economic‑game tasks. Demonstrates graceful performance decline and confirms the resilience of interview‑based agents, with maximal agents remaining the most accurate.