When Generative AI Enters Consumer Research: What Is Changing in Research Methods?

The rapid development of generative artificial intelligence is redefining how consumer and marketing researches are conducted. When a model can replicate the consumer research processes of 35 top-tier journal articles within minutes, are we still engaging in “human-led science”? A study by Yoo, Haenlein, and Hewett, published in the Journal of the Academy of Marketing Science, addresses precisely this question: Is AI merely transforming research tools, or is it reshaping research epistemology?

Generative AI is reshaping consumer research by integrating large language models into academic workflows, from research ideation to theory development

The rapid advancements in AI models have given rise to new avenues for research.

Since the release of ChatGPT, large language models (LLMs) and large multimodal models (LMMs) have rapidly become important tools in academic research and have gradually been embedded across multiple stages of the research process (Marr, 2023). Unlike traditional predictive AI, generative AI can create new content such as text, images, audio, and video, while LLMs, originally focused on text processing, have evolved into models capable of integrating multimodal information (Toner, 2023; OpenAI et al., 2024; Gemini Team et al., 2023). This capability stems from their transformer-based architecture and large-scale data training, which enable the models to understand context, connect information, and generate content (Vaswani et al., 2023). In marketing and consumer research, multimodal capability is particularly critical, as consumer behavior often involves the integration of textual, visual, and situational information, and LMMs can process these data simultaneously to provide a more comprehensive understanding (Yu et al., 2023).

However, despite generative AI becoming a major academic focus, systematic research on how it influences the entire consumer research process remains relatively limited. Existing studies tend to focus on specific applications, such as using AI to generate experimental samples or support theory construction, while lacking a comprehensive analysis of the full research pipeline (Sarstedt et al., 2024; Li et al., 2024; Arora et al., 2024; Goli & Singh, 2024).

Against this backdrop, recent research has begun to systematically examine the role of LMMs in consumer research, covering multiple stages including idea generation, theory development, experimental design, data analysis, and research reporting, while also evaluating their advantages and limitations.

AI Is Changing How Research Questions Are Generated

The first step in consumer research typically involves identifying gaps in the literature and formulating research questions. Traditionally, this process has relied on researchers’ experience, intuition, and extensive reading of prior studies, but generative AI is beginning to reshape this logic. Research suggests that LMMs can generate research ideas by identifying patterns in large-scale data and can simulate “associative thinking” and “divergent thinking” (Mednick, 1962; Guilford, 1950). Unlike human researchers, these models do not rely on intuition but generate hypotheses based on data relationships, which may, to some extent, reduce cognitive biases such as confirmation bias and anchoring bias (Hutson, 2023).

In practice, LMMs can rapidly conduct literature searches, identify potential research gaps, and propose new research questions. For example, they may ask how perceived crowding in virtual shopping environments influences responses to sales promotions, or examine differences in consumer responses to digital versus physical art products under scarcity cues—questions that reflect trends in the platform economy and emerging technologies (Colback, 2023; Yoo et al., 2023).

However, these models also exhibit clear limitations. Their literature reviews often rely on publicly available data, their suggested research gaps tend to converge, and they frequently lack deeper explanations regarding the importance of the proposed research questions.

Large multimodal models are transforming every stage of consumer research, including idea generation, theory development, experimental design, data analysis, and academic reporting

Theory Development: The Potential and Boundaries of Cross-Disciplinary Integration

Theory building lies at the core of consumer research, requiring scholars to explain relationships among variables and develop testable hypotheses. LMMs demonstrate notable strengths at this stage. They can integrate theories from psychology, behavioral economics, and management, and propose variable relationships and hypotheses accordingly (Banker et al., 2024; Eapen et al., 2023). For example, in research on brand stockouts, models may draw on the theory of planned behavior and expectation-disconfirmation theory; in studies of negative brand information, they may invoke cognitive dissonance theory and attachment theory as explanatory frameworks. This cross-disciplinary integration makes AI a valuable tool for theoretical exploration.

However, theory development involves more than linking variables; it also requires understanding causal mechanisms, contextual embeddedness, and depth of explanation, areas that mark the current limits of these models. LMMs do not possess genuine causal reasoning capabilities; their outputs remain largely based on statistical associations and semantic patterns (Pearl & Mackenzie, 2018). Therefore, theoretical frameworks generated by models must be critically evaluated and validated by researchers before they can be transformed into meaningful theoretical contributions for empirical research.

Pretesting and Pilot Testing: The Possibilities and Risks of Silicon Samples

During the research design stage, scholars typically use pretests to validate survey instruments and stimuli. LMMs can simulate participant responses by generating “silicon samples,” thereby reducing costs and increasing efficiency (Demszky et al., 2023). Research shows that models perform relatively well on qualitative questions, producing responses similar to those of human participants, though often with less emotional depth and fewer details. For quantitative questions, their results approximate those of real samples but still exhibit discrepancies. More importantly, multiple studies indicate that models struggle to replicate the diversity of genuine human behavior (Park et al., 2024; Santurkar et al., 2023; Tjuatja et al., 2023). Thus, while silicon samples have instrumental value in the pretesting stage, their results should not be regarded as direct substitutes for real consumer responses; rather, they are better suited for early-stage exploration and design refinement.

Experimental Design and Data Generation: An Efficiency Revolution

In the experimental stage, LMMs can generate survey instruments, experimental scenarios, and virtual consumer environments, and automatically create datasets. Research indicates that models can generate silicon samples based on research designs, randomly assign conditions, and output data, substantially improving research efficiency (Chang et al., 2024; Sarstedt et al., 2024). However, their limitations are equally evident. Issues arise in random assignment, sample representativeness, and biases embedded in training data (Gui & Toubia, 2023; Haim et al., 2024). Furthermore, due to safety constraints, models often avoid generating sensitive or extreme scenarios, which may inadvertently weaken the extent to which experiments capture real-world complexity, making results more “normalized” than realistic. Consequently, LMMs are better understood as supportive tools in experimental design rather than mechanisms that replace real experimental environments.

Data Analysis: Capabilities and Uncertainty Coexist

At the data analysis stage, LMMs have demonstrated certain capabilities. They can explain statistical concepts, recommend analytical methods, and generate code, performing reliably in basic analyses such as correlations, t-tests, and regression (Korinek, 2024; Cooper et al., 2024). However, when more complex statistical structures are involved, such as mediation models, moderation effects, or multilevel models, where their performance remains unstable and may even produce misleading results (Frieder et al., 2023; Ignjatović & Stevanović, 2023).

More critically, analyses based on model-generated data often diverge significantly from the conclusions of the original studies, sometimes even producing effects in the opposite direction in replication exercises. This suggests that LMMs still exhibit substantial uncertainty in statistical inference and data generation, and their analytical outputs require rigorous validation and replication. At present, AI is more appropriately positioned as an analytical support tool rather than an autonomous agent capable of conducting complex statistical reasoning independently.

Reporting: AI as a Tool for Scholarly Expression and Knowledge Dissemination

In the reporting stage, LMMs function not only as language-editing tools but increasingly as systems that support knowledge expression and dissemination. Research indicates that models perform particularly well in correcting grammar, spelling, and punctuation, a capability derived from their large-scale textual training, which enables them to recognize diverse linguistic errors and improve writing quality (Li et al., 2023). More importantly, LMMs can understand context and adjust expression for different audiences, for instance, generating distinct versions of research narratives for academic readers versus managerial audiences, thereby enhancing the readability and communicative reach of research findings (Wood, 2024).

Nevertheless, the reporting stage is also one of the most risk-prone phases. AI-generated text may lack originality or contain inaccurate citations, necessitating careful scrutiny by researchers to ensure academic rigor and research credibility. Moreover, academia faces emerging normative questions, such as how to disclose the use of LMMs in publications, how to define AI’s contribution, and how to maintain transparency and research ethics when using AI-assisted writing. These considerations suggest that while AI can improve writing efficiency, its role should remain that of “assisted expression” rather than a substitute for scholarly judgment.

The future of consumer research lies in human–AI collaboration, where researchers and generative AI work together to enhance insight generation and innovation

Overall: AI Does Not Replace Researchers but Optimizes the Research Process

The true contribution of Yoo and colleagues’ study does not lie in demonstrating that LMMs can replicate 35 published articles; rather, it lies in revealing a deeper question: when the research process can be decomposed into standardized steps, does this imply that academic knowledge production has become highly programmatic? While AI can help researchers restructure research workflows, accelerate literature reviews, support theoretical integration, assist in experimental design, and improve academic writing, its limitations remain clear. Models still exhibit instability in data generation, behavioral simulation, and complex statistical analyses, and the results they produce may at times diverge from those of real empirical studies. Moreover, excessive reliance on these models may lead to the convergence of research questions, reduced innovation, and risks such as “AI hallucinations.” Therefore, future consumer research will require new governance mechanisms, which shall include data validation, research replication, ethical review, and methodological transparency to ensure the reliability and academic rigor of AI-assisted research.

Overall, as technology advances, research paradigms may gradually shift toward human–AI collaboration, in which AI functions as a cognitive tool within the research process while researchers remain responsible for theoretical judgment and methodological design. Under this model, generative AI is no longer merely a tool but becomes an important force driving research innovation and cross-disciplinary integration.

Reference:

Arora, N., Chakraborty, I., & Nishimura, Y. (2024). EXPRESS: AI-Human Hybrids for Marketing Research: Leveraging LLMs as Collaborators. Journal of Marketing. https://doi.org/10.1177/00222429241276529

Banker, S., Chatterjee, P., Mishra, H., & Mishra, A. (2024). Machine-Assisted Social Psychology Hypothesis Generation. American Psychologist, 79(6), 789–797. https://doi.org/10.31234/osf.io/kv6f7

Chang, S., Kennedy, A., Leonard, A., & List, J. A. (2024). 12 Best Practices for Leveraging Generative AI in Experimental Research (Working Paper 33025). National Bureau of Economic Research.

Colback, L. (2023). The rise of the platform economy. Financial Times. https://www.ft.com/content/e5f5e5b9-3aec-439a-b917-7267a08d320f

Cooper, N., Clark, A. T., Lecomte, N., Qiao, H., & Ellison, A. M. (2024). Harnessing large language models for coding, teaching and inclusion to empower research in ecology and evolution. Methods in Ecology and Evolution, 00, 1–7. https://doi.org/10.1111/2041-210X.14325

Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., Jones, M., Krettek-Cobb, D., Lai, L., JonesMitchell, N., Ong, D. C., Dweck, C. S., Gross, J. J., & Pennebaker, J. W. (2023). Using large language models in psychology. Nature Reviews Psychology, 2(11), 688–701. https://doi.org/10.1038/s44159-023-00241-5

Eapen, T. T., Finkenstadt, D. J., Folk, J., & Venkataswamy, L. (2023, July 1). How Generative AI Can Augment Human Creativity. Harvard Business Review. https://hbr.org/2023/07/how-generative-ai-can-augment-human-creativity

Frieder, S., Pinchetti, L., Chevalier, A., Griffiths, R.-R., Salvatori, T., Lukasiewicz, T., Petersen, P. C., & Berner, J. (2023). Mathematical Capabilities of ChatGPT. arXiv. https://doi.org/10.48550/arXiv.2301.13867

Hutson, M. (2023). Hypotheses devised by AI could find ‘blind spots’ in research. Nature. https://doi.org/10.1038/d41586-023-03596-0

Gemini Team, Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., Silver, D., Petrov, S., Johnson, M., Antonoglou, I., Schrittwieser, J., Glaese, A., Chen, J., Pitler, E., … Vinyals, O. (2023). Gemini: A Family of Highly Capable Multimodal Models. arXiv. https://doi.org/10.48550/arXiv.2312.11805

Goli, A., & Singh, A. (2024). Frontiers: Can Large Language Models Capture Human Preferences? Marketing Science, 43(4), 709–722. https://doi.org/10.1287/mksc.2023.0306

Gui, G., & Toubia, O. (2023). The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective. SSRN Electronic Journal, 1–27. https://doi.org/10.2139/ssrn.4650172

Guilford, J. P. (1950). Creativity. American Psychologist, 5(9), 444–454.

Haim, A., Salinas, A., & Nyarko, J. (2024). What’s in a Name? Auditing Large Language Models for Race and Gender Bias. arXiv. https://doi.org/10.48550/arXiv.2402.14875

Ignjatović, A., & Stevanović, L. (2023). Efficacy and limitations of ChatGPT as a biostatistical problem-solving tool in medical education in Serbia: A descriptive study. Journal of Educational Evaluation for Health Professions, 20, 28. https://doi.org/10.3352/jeehp.2023.20.28

Korinek, A. (2024). LLMs Level Up—Better, Faster, Cheaper: June 2024 Update to Section 3 of “Generative AI for Economic Research: Use Cases and Implications for Economists”. Journal of Economic Literature, 61(4).

Li, P., Castelo, N., Katona, Z., & Sarvary, M. (2024). Frontiers: Determining the Validity of Large Language Models for Automated Perceptual Analysis. Marketing Science, 43(2), 254–266. https://doi.org/10.1287/mksc.2023.0454

Li, Y., Sha, L., Yan, L., Lin, J., Raković, M., Galbraith, K., Lyons, K., Gašević, D., & Chen, G. (2023). Can large language models write reflectively. Computers and Education: Artificial Intelligence, 4, 1–11. https://doi.org/10.1016/j.caeai.2023.100140

Marr, B. (2023). The Best Examples Of What You Can Do With ChatGPT. Forbes. https://www.forbes.com/sites/bernardmarr/2023/03/01/the-best-examples-of-what-you-can-do-with-chatgpt/?sh=49c7fbdedf11

Mednick, S. (1962). The associative basis of the creative process. Psychological Review, 69(3), 220.

OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., … Zoph, B. (2024). GPT-4 Technical Report. arXiv. https://doi.org/10.48550/arXiv.2303.08774

Park, J. S., Zou, C. Q., Shaw, A., Hill, B. M., Cai, C., Morris, M. R., Willer, R., Liang, P., & Bernstein, M. S. (2024). Generative Agent Simulations of 1,000 People. arXiv. https://doi.org/10.48550/arXiv.2411.10109

Pearl, J., & Mackenzie, D. (2018). The book of why: The new science of cause and effect. Basic Books.

Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., & Hashimoto, T. (2023). Whose Opinions Do Language Models Reflect? Proceedings of Machine Learning Research, 202, 29971–30004.

Sarstedt, M., Adler, S. J., Rau, L., & Schmitt, B. (2024). Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines. Psychology & Marketing, 41, 1254–1270. https://doi.org/10.1002/mar.21982

Tjuatja, L., Chen, V., Wu, S. T., Talwalkar, A., & Neubig, G. (2023). Do LLMs exhibit human-like response biases? A case study in survey design. arXiv. https://doi.org/10.48550/arXiv.2311.04076

Toner, H. (2023). What Are Generative AI, Large Language Models, and Foundation Models? Center for Security and Emerging Technology. https://cset.georgetown.edu/ article/what-are-generative-ai-large-language-models-and-foundation-models/

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2023). Attention Is All You Need. arXiv. https://doi.org/10.48550/arXiv.1706.03762

Wood, S. (2024). The future of consumer research methods: Lessons of a prospective retrospective. Journal of Consumer Research, 51(1), 151–167.

Yoo, K., Welden, R., Hewett, K., & Haenlein, M. (2023). The merchants of meta: A research agenda to understand the future of retailing in the metaverse. Journal of Retailing, 99(2), 173–192. https://doi.org/10.1016/j.jretai.2023.02.002

Yu, W., Yang, Z., Li, L., Wang, J., Lin, K., Liu, Z., Wang, X., & Wang, L. (2023). MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities. arXiv. http://arxiv.org/abs/2308.02490