AI in Qualitative Research: A Collaborative Partner or Just "Chatting"?

AI in Qualitative Research: Collaborative Partner or Epistemic Risk?

The rise of Generative Artificial Intelligence (GenAI) has sparked a profound scholarly debate in the field of qualitative research, centered on a critical question: Do Large Language Models (LLMs) enhance the depth of qualitative exploration, or do they fundamentally threaten its epistemological foundations? Adam S. Hayes (2025) argues that LLMs are revolutionizing how scholars work with textual data by allowing them to "converse" with their materials, moving beyond traditional codebooks or manual line-by-line analysis through targeted questioning, probing for contextual insights, and refining theoretical connections. In this view, the LLM becomes an active partner that identifies patterns that might otherwise take weeks of manual work, freeing researchers for deeper theoretical reflection and meaning-making. Hayes emphasizes that LLMs demonstrate a form of language "understanding" that enables them to grasp semantic relationships, contextual meanings, and subtle linguistic patterns across multiple scales of analysis (Brown et al., 2020; Ziems et al., 2024).

LLMs, Coding, and Meaning-Making in Qualitative Data Analysis

However, Duc Cuong Nguyen and Catherine Welch (2026) offer a sharp counterpoint, asking in their title: "Analyzing—Or Just Chatting?". They warn that the uncritical adoption of this technology by management researchers will introduce unacceptable epistemic risks. Unlike Hayes's emphasis on LLMs as collaborative partners, Nguyen and Welch argue that using LLMs for qualitative data analysis constitutes a "category error," mistaking a synthetic, probability-based predictive next-word generator for an analytical aid. They contend that LLM outputs are merely word strings selected probabilistically, and while they may resemble human creativity and reasoning, the technology is fundamentally "indifferent to the truth" of its outputs (Hicks et al., 2024; Narayanan & Kapoor, 2024).

The two papers diverge significantly on the core issue of whether AI possesses true understanding. Hayes believes that LLMs have a unique ability to grasp complex linguistic features such as tone, cultural connotations, contextual shifts, and implicit meaning—effectively "reading between the lines". He describes a "three-way conversation" between the researcher, the data, and the LLM, where the LLM serves as an intelligent mediator grounded in extensive training across massive textual corpora (Brown et al., 2020; Chowdhery et al., 2022). This capability allows researchers to move seamlessly between micro-level textual details and macro-level theoretical frameworks.

In contrast, Nguyen and Welch maintain that this perceived understanding is an "anthropomorphic fallacy" driven by the model's design. They explain that rather than processing raw data or performing systematic retrieval, an LLM reduces and transforms data into vector embeddings to generate statistically plausible responses. They specifically point out that the randomness and instability of the technology trap researchers in an "infinite loop of chatbot conversations," where the need to constantly validate accuracy, contextual relevance, and connections to the data negates any supposed efficiency gains. They argue that these automated features conceal the absence of analytical engagement and distort the researcher’s ability to uncover meaningful insights.

From Conversing with Data to Just Chatting The GenAI Debate

Ultimately, this debate highlights the role of the human in qualitative inquiry. Hayes argues that LLMs do not diminish the centrality of human expertise; the researcher remains the sense-maker, responsible for selecting prompts, verifying outputs, and situating them within conceptual frameworks. He sees LLMs as a catalyst for deeper investigation rather than a one-click solution. Nguyen and Welch, however, worry that the enthusiasm for AI efficiency and the misplaced trust in LLMs as "neutral experts" may lead to a homogenization of knowledge production and marginalize traditions, such as critical, interpretive, and reflexive methodologies, that are vital for rich qualitative insights (Köhler et al., 2022; Mees-Buss et al., 2022). This scholarly "contest" serves as a reminder that while embracing innovation, we must critically reflect on how our tools shape our observations and uphold the core mission of interpreting social meaning.

References:

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language Models are Few-Shot Learners. arXiv.Org. 

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., … Fiedel, N. (2022). PaLM: Scaling Language Modeling with Pathways. arXiv.Org.

Hayes, A. S. (2025). “Conversing” With Qualitative Data: Enhancing Qualitative Research Through Large Language Models (LLMs). International Journal of Qualitative Methods, 24. https://doi.org/10.1177/16094069251322346

Hicks, M. T., Humphries, J., & Slater, J. (2024). ChatGPT is bullshit. Ethics and Information Technology, 26(2), Article 38. https://doi.org/10.1007/s10676-024-09775-5

Köhler, T., Smith, A., & Bhakoo, V. (2022). Templates in Qualitative Research Methods: Origins, Limitations, and New Directions. Organizational Research Methods, 25(2), 183–210. https://doi.org/10.1177/10944281211060710

Mees-Buss, J., Welch, C., & Piekkari, R. (2022). From Templates to Heuristics: How and Why to Move Beyond the Gioia Methodology. Organizational Research Methods, 25(2), 405–429. https://doi.org/10.1177/1094428120967716

Narayanan, A., & Kapoor, S. (2024). AI snake oil: what artificial intelligence can do, what it can’t, and how to tell the difference. Princeton University Press. https://contents.bibs.aws.unit.no/files/images/large/1/3/9780691249131.jpg

Nguyen, D. C., & Welch, C. (2026). Generative Artificial Intelligence in Qualitative Data Analysis: Analyzing—Or Just Chatting? Organizational Research Methods, 29(1), 3–39. https://doi.org/10.1177/10944281251377154

Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024). Can Large Language Models Transform Computational Social Science? Computational Linguistics - Association for Computational Linguistics, 50(1), 237–291. https://doi.org/10.1162/coli_a_00502