AI in Research: Real-World Stimuli, Synthetic Faces, and the Validity Question
Image via Wikimedia Commons (public domain).
Welcome to this week's scan of how AI is reshaping research — curated from Nature, PNAS and the leading business and economics journals.
Generative AI is changing what an experimental stimulus can be
Behavioural researchers have long lived with the stimulus-sampling problem: experiments rely on a handful of stylised stimuli, so it is hard to know whether effects generalise beyond them. In the Journal of Marketing Research, Anirban Mukherjee, Hannah H. Chang and Sachin Gupta introduce BRIDGE, a method that uses foundational generative-AI embeddings to build low-dimensional, interpretable representations of the focal construct plus statistical controls for nuisance variation in unaltered real-world texts. Monte Carlo simulations, two coffee-certification experiments and a large choice experiment show it recovers true parameters even when the texts contain unobserved nuisance variation. In the choice study, 1,000 participants evaluated roughly 50,000 unique product descriptions drawn from a corpus of nearly 120,000 — and entirely incidental initial products shaped later preferences. The authors ship a researcher's guide and a Python package.
AI can also go the other way: instead of taming messy real stimuli, it can generate precisely controlled ones. In a Marketing Science Frontiers paper, Lan Luo and Olivier Toubia run a field experiment with AI-generated headshots that vary only in femininity and find that consumers discriminate against femininity independently of gender identity — a separation that is very hard to achieve with photos of real people. Yet the more our evidence is built from machine-made material, the sharper the concern raised by Jacob Teeny, Andrew Luttrell and Dongchan Lee in PNAS, whose piece names a "synthetic validity" problem in AI-generated science.
Why it matters: For experimental social scientists and marketers, generative AI now offers two powerful levers — realism at scale (BRIDGE) and surgical control over stimuli (synthetic headshots). Both can strengthen causal claims, but only if researchers validate that AI-mediated stimuli and representations measure what they claim to. Treat the new tools as design upgrades that come with a validation burden, not as shortcuts.
More from this week
A research agenda for computational creativity
Min Ding, Ram Gopal, Ulrike Schultze and D. J. Wu define creativity as a structured, instrumentable process of search, recombination, evaluation and realization. Drawing on five special-issue articles, they argue generative AI's promise lies in configuring human and machine agency so outputs are novel and useful while creative work stays effortful, accountable and socially meaningful. The agenda spans process instrumentation, diversity-preserving design, agency-sensitive governance and organizational capability building.
Prediction rules that survive gaming
When consequential decisions are encoded in machine-learning algorithms, people have incentives to change their behaviour to get the outcome they want. This American Economic Review paper develops an empirical approach that explicitly models those incentives and adjusts decision rules to anticipate manipulation, producing rules that remain stable even when fully transparent. It is directly relevant to anyone designing or studying scoring, screening or targeting algorithms.
When 'honest' causal forests hurt
Causal forests are widely used to estimate heterogeneous treatment effects for personalised interventions in marketing, operations and policy. Standard practice is honest estimation — one sample defines subgroups, another estimates effects — and it is the default in many software packages. This Information Systems Research paper shows honest estimation can reduce accuracy in some settings, so researchers should treat it as a choice rather than an automatic setting.
AI and perception biases in investing
A new experimental study in the Journal of Financial Economics by Anastassia Fedyk, Ali Kakhbod, Peiyao Li and Ulrike Malmendier examines how AI interacts with perception biases in investment decisions. It adds to a growing body of behavioural-finance work testing whether AI advice corrects or reinforces human judgment errors — worth a look for anyone studying AI-assisted decision-making.
AI formalizes the proof of Fermat's Last Theorem
Nature's news team reports that an Anthropic AI system has 'formalized' the proof of Fermat's Last Theorem, calling it a milestone for mathematics. Formalization turns a human proof into machine-checkable form, one of the most demanding tests of rigour in any field. It is a striking signal of how quickly AI is moving from assisting with research to verifying it.
Use AI as a sparring partner, not an oracle
In a Nature piece, Marc Schmitt argues that researchers get the most from AI when they use it as a sparring partner to challenge their thinking rather than as an oracle whose answers are taken on trust. It is a useful framing for anyone setting norms for AI use in their lab or classroom.
That's this week. Forward it to a colleague who's still running experiments with three stimuli — and explore AI tools for your own research at gaiforresearch.com.
Comments