AI in Research: When Your Model Has Already Seen the Future
Image: "New York Stock Exchange August 2017 02" by Arild Vågen (CC BY-SA 4.0) via Wikimedia Commons, recoloured.
Welcome to this week's scan of how AI is reshaping research — curated from Nature, Science, PNAS and the leading business journals.
Leakage: the quiet validity threat in AI-powered research
Social scientists increasingly feed news, filings and reviews into large language models to measure sentiment, classify text or forecast outcomes. The problem, as Songrun He, Linying Lv, Asaf Manela and Jimmy Wu note in the Journal of Financial Economics, is that a model trained on today's internet has, in effect, already read the future: its training data can introduce lookahead bias and training leakage into any historical analysis. Their fix is to train a suite of chronologically consistent models — ChronoBERT and ChronoGPT — that use only text available at each point in time. Despite that strict constraint, the models match or outperform widely used models such as BERT on standard benchmarks and remain competitive with larger open-weight models. In an asset-pricing test predicting next-day stock returns from financial news, their real-time outputs deliver Sharpe ratios comparable to a much larger, potentially biased Llama model. A subtle takeaway: lookahead bias is model- and application-specific, so it has to be checked case by case rather than assumed away.
Leakage is also a problem when AI does the research itself. In PNAS, Ziming Luo, Atoosa Kasirzadeh and Nihar B. Shah systematically probe open-source 'AI scientist' systems for four pitfalls: inappropriate benchmark selection, data leakage, metric misuse and post hoc selection bias. Controlled experiments isolating each pitfall reveal systematic vulnerabilities across two representative systems. Crucially, these flaws are largely invisible in the final manuscript — the very artefact peer review evaluates. The authors show that access to full workflow artefacts, such as log traces and code, makes auditing considerably more effective, and they call on journals, conferences and researchers to move from manuscript-only review toward process auditing.
Why it matters: Whether an LLM is your measurement instrument or your research assistant, the same question applies: what did the system know, and when? For empirical work in marketing, finance and management, that means documenting model training cut-offs relative to your sample period, preferring time-consistent models for any backtest or forecasting claim, and keeping (and sharing) the logs and code behind AI-assisted analyses. Reviewers who see only a polished PDF cannot catch what these papers show is easy to miss.
More from this week
An AI agent resolves open Erdős problems — with machine-checked proofs
LLMs are increasingly good at maths but unreliable, which limits their use in real research. This team built an agent that generates formal proofs in Lean, where a compiler verifies every step. It autonomously resolved nine of 353 open Erdős problems and proved 44 of 492 conjectures from the On-Line Encyclopedia of Integer Sequences — and even a basic agent alternating LLM generation with Lean verification replicated the Erdős results. The broader lesson: AI output becomes far more trustworthy when it is paired with an independent, automatic checker.
Data scientists vs. domain experts: why AI predictions go unused
Predictive models often remain inscrutable to the experts expected to use them. Drawing on an 18-month inductive study of three large Indian banks, Joshi, Su and Zbaracki trace the problem to competing epistemic stances over how knowledge should be produced and evaluated: data scientists produce new knowledge, but domain experts decide whether it is deployed. Direct attempts to resolve the conflict often failed; what worked were four bridging practices that embedded predictions in the boundary objects and routines experts already use — a useful lens for anyone studying AI adoption in organisations.
ChatGPT's release moved stock prices by workforce exposure
Eisfeldt, Schubert, Zhang and Taska construct a measure of firms' workforce exposure to generative AI and show that a portfolio long highly exposed and short less exposed firms earned 5% in the two weeks after ChatGPT's release. The effect was stronger for firms with more data assets and was distinct from firms' product exposure to AI. Evidence on subsequent labour demand and profitability points to a labour–technology substitution channel.
From data-model fit to model-world fit
ML systems often perform well on training data yet fail once deployed. Castellanos, Tremblay, Lukyanenko, Parsons and colleagues argue the field over-optimises data-model fit and propose a 'model-world fit' framework, in which domain knowledge enters development through external representations such as process models or ontologies to address know-what, know-how and know-why gaps. They illustrate it with child welfare placement, where statistical accuracy alone is an insufficient evaluation criterion.
Retrofitting language models to read raw bytes
Most LLMs read text as subword tokens, which obscures character-level detail — a real limitation for code, biological sequences and other data where individual characters matter. This Nature paper introduces 'byteification', a two-stage procedure that converts existing subword models into byte-level models with minimal additional training. The resulting models outperform earlier byte-level approaches, excel at character-level reasoning tasks and keep practical inference speeds while reusing the source model's ecosystem.
That's this week. Forward it to a colleague who's running LLMs over historical text — and explore AI tools for your own research at gaiforresearch.com.
Comments