AI in Research: Can You Still Check What the Machine Made?
Image: "Microscopy Lab Tour at the Colorado School of Mines - 8905720941" by SparkFunElectronics (CC BY 2.0) via Wikimedia Commons, recoloured.
Welcome to this week's scan of how AI is reshaping research — curated from Nature, Science, PNAS and the leading business and social-science journals.
Provenance and contestability: can we still check AI-made science?
In a new Science piece, Christoph Trattner argues that AI is "moving upstream" in science. Beyond tools like AlphaFold, agentic systems can now search literatures, generate hypotheses, write and run code, interpret results and automate much of the path from idea to paper. The deeper shift, he contends, is in who — or what — decides which questions are asked, which evidence is surfaced and which alternatives survive. As machine intelligence becomes part of science's epistemic infrastructure, researchers who did not build, own or operate a system must still be able to contest how ideas and evidence became a claim.
Two stories this week show what that looks like in practice. Google DeepMind researchers report in Nature on SynthIDBio, which embeds a hidden watermark into AI-designed protein sequences and structures while preserving function: watermarked protein binders kept binding affinity comparable to unmarked ones, with near-perfect detection. As Nature's news coverage notes, though, the mark can be scrubbed by running a protein through another design tool, so it is one layer of provenance rather than a guarantee. Meanwhile, the winning video of Nikon's Small World in Motion competition drew criticism from microscopists who say it shows biologically implausible structures. The organiser later added that an "unsupervised" AI model assisted with post-processing; the creator says AI was used only to distinguish and colour structures, not to generate the cilia or their motion. As one researcher told Nature, scientific images "are data".
Why it matters: For social scientists and business researchers the lesson travels well beyond biology. Whenever an AI step sits between your data and your claim — LLM-coded transcripts, AI-cleaned survey responses, synthetic stimuli, model-generated figures — reviewers and readers need to be able to see and challenge that step. Disclose where AI touched the pipeline, keep the raw inputs, and treat "the model did it" as a method that must be documented and contestable, not a black box.
More from this week
AlphaEvolve tackles 67 maths problems — with Terence Tao
Bogdan Georgiev, Javier Gómez-Serrano, Terence Tao and Adam Zsolt Wagner use AlphaEvolve, an evolutionary coding agent that pairs LLMs with automated evaluation, on 67 problems across analysis, combinatorics, geometry and number theory. It rediscovered the best known solutions in most cases, improved on them in several, and sometimes generalised results for finitely many inputs into a formula valid for all inputs. The key design feature is the automated evaluator: every proposal is checked, which is exactly the kind of verifiability most research domains lack.
Generative search raises purchases in a large field experiment
Shuang Zheng and co-authors ran a large-scale field experiment on Meituan, a major Chinese platform, comparing generative search (AI-generated responses shown before conventional results) with keyword search. Generative search significantly increased purchases, produced more effective and diverse queries, and reduced exploratory browsing while concentrating evaluation on relevant categories and merchants. For marketing researchers, it makes a previously hidden stage of search — turning a need into keywords — empirically observable.
Does AI accelerate innovation? 32 researchers weigh in
Drawing on 32 interviews with U.S. academic manufacturing and materials-science researchers experienced with AI/ML, John Nelson and colleagues find AI mainly used to model materials and processes, enabling cheaper, faster search of design spaces. But the tools were unreliable outside data-rich regions, required skilled use alongside older techniques, and raised worries about bypassing disruptive theoretical advances. The authors see grounds for optimism about incremental innovation, provided conventional research is still funded.
'Epistemic enchantment': how AI hiring projects keep growing
Elmira van den Broek and Natalia Levina followed an HR department introducing AI for candidate screening over three years. Datafication expanded far beyond the technology's original scope, which they explain through epistemic enchantment: members became captivated by AI's promise of predictive, objective, ever-improving insight while overlooking its costs and risks. 'Ideal data' became an ever-desired but never-reached goal — a useful concept for anyone studying, or running, AI adoption.
That's this week. Forward it to a colleague who's wondering how to disclose the AI in their pipeline — and explore AI tools for your own research at gaiforresearch.com.
Comments