Coding Agents for Reproducible Data Analysis: Claude Code, Codex and Gemini CLI
Image via Wikimedia Commons (public domain).
Until recently, using AI for data analysis meant pasting code into a chat window and pasting the answer back. In 2025 the tools changed shape. Coding agents such as Claude Code, OpenAI Codex and Gemini CLI run inside your project folder, read your data and scripts, write and execute code, inspect the output and fix their own mistakes. For researchers this is a bigger change than better autocomplete: the agent can carry out an entire analysis from raw data to figure, and you can watch it do so. This tutorial explains what the tools are, how to set up a project so that the result is reproducible, and where to keep a human in the loop.
What the three tools are
Claude Code (Anthropic) runs in the terminal, in a desktop app or inside VS Code and JetBrains. It is strong at understanding an existing codebase and at long, multi-step tasks.
Codex (OpenAI) lives inside ChatGPT and works in an isolated cloud copy of your repository, proposing changes as pull requests. A command-line version also exists.
Gemini CLI (Google) is an open-source terminal agent with a 1 million token context and a generous free tier for personal Google accounts.
All three can be pointed at a folder containing a CSV file and a README and asked to "clean the data, run the pre-registered regression and produce Table 2". The difference from a chat window is that they will actually run the code, see the error messages, and iterate.
Set the project up for reproducibility first
An agent will happily produce an analysis that runs once on your machine and never again. Before the first prompt, put four things in place.
A folder with version control. Initialise a git repository. Every change the agent makes is then a diff you can read, accept or revert.
A pinned environment. A requirements.txt or renv.lock with exact versions. Ask the agent to install into a virtual environment and to record every package it adds.
A project instructions file. Claude Code reads CLAUDE.md, Codex reads AGENTS.md, Gemini CLI reads GEMINI.md. Write down the research question, the pre-registered analysis, the variable definitions, and rules such as "never modify the raw data folder" and "every figure is produced by a script in figures/".
Raw data that is read-only. Keep the original files in a folder the agent is told not to touch, and have every script write to a separate output folder.
A worked prompt
A good first task is narrow and checkable. For example: "Load data/raw/survey.csv. Report the number of rows, the number of missing values per column, and the distribution of the condition variable. Do not modify anything yet." Read the output. Then: "Write scripts/01_clean.py that applies the exclusion criteria in CLAUDE.md, saves the result to data/clean/survey.parquet, and prints how many rows were removed by each criterion." Ask for the exclusion counts to be written to a text file you can cite. Only after you have checked those counts should you move on to the model.
Where to keep the human
The agent is good at plumbing and bad at judgement. Decide yourself which variables enter the model, which tests are pre-registered and which are exploratory, and how outliers are handled. Read every script the agent writes; they are usually short. Rerun the whole pipeline from raw data in a fresh environment before submission, ideally with a single command, and put that command in the README. Disclose the tool and model version in the methods section, as most journals now require.
Cost and access
Claude Code needs a Claude Pro or Max subscription or API credits. Codex is included with ChatGPT Plus and Pro. Gemini CLI is free for personal accounts with daily limits that are ample for most analyses. Institutional licences are available from all three vendors and usually come with data-handling terms that consumer accounts lack; check with your IT department before putting participant data into any of them.
Links to each tool are on our coding and data analysis toolkit page.
Comments