AI in Research: Machine-Made Proofs and the Trust Problem
Image via Wikimedia Commons.jpg) (public domain).
Welcome to this week's scan of how AI is reshaping research, curated from Nature, Science, PNAS and the leading business journals.
AI is doing frontier mathematics. The open question is verification
Mathematics has become the proving ground for AI-driven discovery, and this week three pieces show both the promise and the problem. In PNAS, Yichen Huang starts from OpenAI's disproof of the Erdős unit distance conjecture, a milestone that, he notes, relied on an internal model with undisclosed test-time compute and so could not be independently reproduced. His response is a deliberately simple autonomous agent built on the publicly available GPT-5.5 Pro, using a problem-agnostic three-stage prompting pipeline: brainstorm, execute, review. Across eight independent trials with web search disabled, every run produced a correct or essentially correct disproof, the proofs were diverse rather than copies of earlier AI output, and each trial used on average about 107k reasoning tokens. Because the model was released nearly a month before OpenAI published its result, the test avoids data contamination. Code and all outputs are released.
That openness matters because the pace has outrun the checking. A Science news feature reports that as AI companies churn out math breakthroughs, their proof-checking systems are coming under scrutiny. In an accompanying Science editorial, editor-in-chief H. Holden Thorp describes the scale: OpenAI's announced potential solution to the Navier-Stokes Millennium Prize problem, produced by an army of AI agents at a cost of millions of dollars in compute, followed weeks later by 722 manuscripts on 372 problem families and a DeepMind math system's solutions published in Science. Thorp's question reaches beyond mathematics: can the satisfaction of understanding ever replace the thrill of discovery?
Why it matters: The same pattern is coming to every field. When AI can produce results faster than humans can verify them, credibility shifts to whoever makes the work reproducible: public models, disclosed compute, released code and outputs. Huang's paper is a template for social scientists using LLMs too. Report the model version, the prompting pipeline, the number of runs and how often it worked, and share the transcripts, so others can check rather than simply trust.
More from this week
A fix for LLM 'silicon samples' that don't match humans
LLMs are increasingly used to measure human preferences and opinions, but their answers can diverge from real respondents. Yan Leng, Yunxin Sang and Ashish Agarwal introduce human mimicry calibration (HuMCal), a post hoc, model-agnostic method that uses aggregate human benchmark data to learn how to weight LLM outputs from different prompts, personas and model checkpoints. In applications to consumer transaction data and public opinion surveys it beat standard aggregation rules, and the calibrated weights often stayed useful across time, geography, industry and question clusters. Directly relevant to anyone experimenting with synthetic respondents.
Who is liable when AI causes harm? Follow the objective function
Martin Hilbert observes that early US lawsuits are holding AI companies accountable via product liability and agency law, while Section 230 and First Amendment defences begin to crack. He argues these routes share one principle rooted in how machine learning works: every system starts from a human-specified objective function, with further layers of deliberate human intent in training data, deployment and oversight. Accountability then becomes a tractable legal question of tracing design decisions, grading each actor's intent and foreseeability, and apportioning liability, without needing AI personhood or speech rights. He flags four remaining gaps, from many-hands problems to harms that surface only after deployment.
The 'intelligence loop': why some AI agents get smarter on the job
Yong Li, Ming Zeng and Alexis Li propose a supply-side counterpart to data network effects: delegated use of an AI agent generates operational experience, which can be converted into future capability, which in turn sustains usage. The loop only starts once an agent crosses a workflow's 'minimum viable intelligence', and even then it can decay, hold steady or accelerate. Their implications: advantage may depend more on the loop than on installed base, productive capacity can outgrow expert headcount, and cutting evaluation or oversight to scale faster may lift output short-term while weakening the loop.
Explainable AI joins the hunt for real Rembrandts
Andrés Arias-Rosales, Arthur K. Wheelock, J. Kenneth Moritz and Philip R. LeDuc present an explainable AI approach designed to complement, not replace, connoisseurs in attributing paintings. It reaches up to 97.9% recall and 95.8% accuracy when highly confident, highlights the image regions that drive each judgement, and is calibrated to avoid overconfidence. The training set deliberately includes comparable artists such as a master's pupils and teachers. A useful model of human-AI collaboration for any field built on expert judgement.
Mapping competitors, public and private, from website text
Gerard Hoberg, Craig A. Knoblock, Gordon M. Phillips and Jay Pujara apply representation learning to company website content, focusing on industry-specific products and services, to place public and private firms in a single embedding space. The result is a peer competitor network that supports analysis of company interactions and market evolution, extending text-based industry classification beyond firms that file public disclosures. A ready-made method for strategy and marketing researchers who need competitor sets for private companies.
That's this week. Forward it to a colleague who's still copy-pasting into ChatGPT, and explore AI tools for your own research at gaiforresearch.com.
Comments