The Wolf is Coming? Social Science and AI Agents in the Era of "Vibe Researching"

Human–AI Delegation in Social Science Codifiable Tasks vs. Tacit Knowledge, Theoretical Originality, and Field Judgment

In an age where everyone is talking about AI, social science research finds itself at a delicate crossroads. Recently, Yongjun Zhang (2026) from Stony Brook University raised a provocative question in his latest paper, "Vibe Researching as Wolf Coming": what parts of your research can only you do, and what can the machines never replace? Is it running tedious code or writing boilerplate literature reviews? Following Andrej Karpathy's 2025 introduction of "vibe coding", where users describe what they want and AI writes the code, Zhang identifies a parallel phenomenon in academia: "vibe researching" (Karpathy, 2025; Zhang, 2026).

Waves of Automation: From Calculators to "Aeroplanes for the Mind"

Automation in social science is not new, and Zhang (2026) classifies its evolution into four distinct waves. From the 1970s-90s, statistical software automated computation; the 2000s saw API-driven data collection; and the 2010s utilized machine learning for text analysis. Throughout these stages, the core of reasoning, design, and theory-building remained firmly human.

However, the fourth wave, beginning around 2024, represents a qualitative shift: reasoning itself is being automated. If the personal computer was a "bicycle for the mind," today’s AI agents are "aeroplanes for the mind". They amplify cognitive reach, but the risks of error are proportionally greater. Unlike chatbots, AI agents are persistent, have file access, and can execute entire research pipelines autonomously.

Evolution of Automation in Social Science_Statistical Software, APIs, Machine Learning, and AI Agents

Deep Dive: Scholar-Skill and the "Evolving" AI Scientist

To illustrate this transformation, Zhang (2026) introduces scholar-skill, a plugin for Claude Code that integrates 26 specialist AI skills into a complete research pipeline. The most striking aspect of this system is its "full-stack" coverage. For instance, the scholar-idea module stress-tests research questions through a five-agent panel (theorist, methodologist, domain expert, editor, and devil’s advocate). For causal identification, it selects the optimal strategy from 13 options and generates code in both R and Stata.

It is important to note that the logic of AI-driven discovery is becoming increasingly sophisticated. The EvoScientist framework, recently proposed by Lyu et al. (2026), further demonstrates this potential: AI agents are no longer just running static pipelines; they are learning through "evolution." By establishing an automated feedback loop, EvoScientist allows AI scientists to adapt their ideas and code-generation strategies based on accumulated interaction histories, thereby avoiding repeated failures or infeasible research directions (Lyu et al., 2026). In essence, the "aeroplane for the mind" described by Zhang is not just flying; it is upgrading its own engines mid-flight through self-iteration.

Drawing the Line: Where AI Struggles

Faced with such powerful, evolving systems, what remains the researcher's distinctive contribution? Zhang (2026) proposes a framework where the redistribution of power is governed by two dimensions: Codifiability and Tacit knowledge requirement.

In this new landscape, execution tasks, such as running regressions or data cleaning, have become areas where AI agents excel because these tasks are highly codifiable and verifiable. However, formulation and planning tasks remain the AI's primary weakness; while systems like EvoScientist have begun attempting to evolve research directions (Lyu et al., 2026), AI still struggles to identify genuinely new social mechanisms or make paradigm-shifting leaps, such as Granovetter's "strength of weak ties" (Zhang, 2026).

The critical insight is that the boundary of delegation does not fall between stages but rather cuts through every single stage. At every step, there is tacit knowledge required, such as understanding field politics, editorial preferences, or the political sensitivity of a design, that only a human can provide (Zhang, 2026).

Fragile Augmentation and Potential Crisis

While AI agents offer substantial gains, this augmentation is "fragile." A "verification gap" emerges: if you never run the models or read the literature yourself, do you still have the cognitive capacity to detect the subtle but fatal errors in AI-generated output? The author suggests the answer is no (Zhang, 2026). Even as AI improves its success rate through multi-agent collaboration and strategy evolution (Lyu et al., 2026), human critical review remains the final defense against systemic bias.

Furthermore, we face a stratification risk from the "AI productivity premium." Beyond subscription fees, barriers like technical skills, English-language bias in data, and varying coverage across research fields could exacerbate inequality in academia.

AI Agents as “Aeroplanes for the Mind”_Human Oversight, Verification Gap, and the Future of AI-Augmented Social Science Research

Responsible Vibe Researching: Five Principles

"The wolf is indeed at the door; in fact, it’s already inside." To avoid losing ourselves, the author proposes five principles for responsible use:

• Disclose: Explicitly report AI assistance in the methods section.

• Verify: All AI-generated code and prose must be reviewed by humans before publishing.

• Maintain Skills: Deliberately practice tasks without AI to preserve judgment.

• Protect Originality: Ensure the core theoretical contribution remains the researcher's own.

• Design for Access: Prioritize open models and shared tools to prevent academic stratification.

The future of social science should not be a blind pursuit of "vibes," but rather a system of governance—much like aviation, that preserves human judgment at critical decision points while utilizing automation to expand our reach.

Reference:

Karpathy, A. (2025, February 6). Vibe coding [Post]. X.

Lyu, Y., Zhang, X., Yi, X., Zhao, Y., Guo, S., Hu, W., Piotrowski, J., Kaliski, J., Urbani, J., Meng, Z., Zhou, L., & Yan, X. (2026). EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery. arXiv.Org.

Zhang, Y. (2026). Vibe Researching as Wolf Coming: Can AI Agents with Skills Replace or Augment Social Scientists? arXiv.Org.