A reproducibility checklist for computational research

The uncomfortable truth about reproducibility is that it is almost never lost at the end. It is lost in a hundred small, forgettable moments while the work is live: the analysis script edited in place until nobody remembers which version produced Figure 3, the dataset quietly re-exported with a filter that changed two rows, the package upgraded on a Tuesday that shifted a result by a decimal. By the time a reviewer or a colleague asks “can you reproduce this?” the answer has usually already been decided, months earlier, by whether a few habits were in place. This is a checklist of those habits — the commitments that make computational work reproducible not through a heroic effort at submission, but as a byproduct of how it was done.

None of what follows requires a specialized reproducibility platform. It requires deciding, early, that the record matters as much as the result — the same structured-credibility instinct that makes a team verify its references instead of trusting its memory.

1. Pre-register the analysis before you see the results

The single most powerful reproducibility habit is also the cheapest: write down what you are going to do before you do it. A pre-registration — even a short one, even an unofficial one committed to your own repository with a timestamp — separates the hypotheses you set out to test from the ones the data suggested afterward. Both are legitimate science, but they are different kinds, and a reader deserves to know which is which.

A workable pre-registration states the hypotheses, the primary outcome, the planned analysis and how it handles missing data and outliers, and the stopping rule for data collection. The value is not bureaucratic. It is that when a result comes out clean and significant, you can show it was the test you planned rather than the one that happened to work — and when an analysis genuinely evolves, the honest paper reports the change instead of quietly presenting the exploratory as confirmatory. If formal registration on a public registry suits the work, use it; if not, a dated, read-only document in the project is far better than nothing.

2. Freeze and checksum the data

A result is only reproducible against a specific state of the data, and “the dataset” is rarely as fixed as it feels. Files get re-exported, a cleaning step gets rerun with a tweaked parameter, a collaborator sends a corrected version — and suddenly two analyses that should match do not, for reasons no one can reconstruct. Two habits close this gap.

• Freeze the analysis dataset. Once cleaning is done, write the exact data your results run on to a read-only, versioned artifact and point the analysis at that, not at a live or regenerated file. The raw data and the cleaning script remain, of course — but the numbers in the paper trace to one frozen snapshot.
• Checksum it.Record a hash (an SHA-256 is plenty) of that frozen file alongside your results. A checksum is a thirty-second habit that answers, definitively and forever, “is this the same data the paper used?” If the hash matches, the bytes are identical; if it does not, you have caught a silent change before it became an irreproducible result.

The same principle extends to any large or sensitive data you cannot redistribute: you may not be able to share the file, but you can always share its checksum and the script that transforms it, so that anyone with legitimate access can confirm they hold exactly what you analyzed.

3. Make the code available — and runnable

Code availability has quietly become an expectation rather than a courtesy; a growing share of journals and funders ask for it directly, and reviewers increasingly look. But there is a difference between code that is available and code that is runnable, and only the second delivers reproducibility. Availability is a repository link. Runnability is a reader being able to clone that repository and regenerate your results without emailing you.

The bar is lower than it sounds. Deposit the analysis code in a public repository and mint a persistent identifier — a DOI via an archive such as Zenodo — so the citation points to a fixed release rather than a moving branch. Include a short README that names the entry point (“run make figures” or “run analysis/main.py”), and make sure a scripted path runs the pipeline end to end. The test to apply before submission: could a competent stranger, given only your repository and your data, reproduce your main figure? If the honest answer is no, that is the gap to close.

4. Pin the computational environment

The most maddening reproducibility failures are the ones where the code is right, the data is right, and the result is still different — because the software underneath moved. A library’s default changed between minor versions, a random-number implementation was updated, a dependency three levels down shifted behavior. Code without a pinned environment is a recipe with no oven temperature.

• Record exact versions. Capture the full dependency tree with exact version numbers — a lockfile, a requirements.txt with pins, an environment.yml, a renv.lock. “Pandas” is not a version; “pandas 2.2.1” is.
• Set and report every seed. Any stochastic step — a train/test split, a bootstrap, a model initialization — needs a fixed random seed, and that seed belongs in the record. Unseeded randomness is the most common reason two runs of identical code disagree.
• Capture the whole environment where it matters.For work sensitive to the full stack, a container image or an environment specification pins not just your libraries but the interpreter and system dependencies around them. It is the difference between “it worked on my machine” and “it works on any machine.”

5. State limitations honestly

Reproducibility is usually filed under mechanics — data, code, environments — but its final component is a matter of writing, and it is the one most often skipped. A result that is perfectly reproducible can still mislead if the paper oversells what it shows. An honest limitations section is part of the reproducibility contract: it tells the reader the conditions under which the finding holds and the conditions under which it might not.

Concretely, that means naming the boundaries of the sample and being explicit about how far the result generalizes; flagging analytic choices that a defensible alternative could have made differently, and ideally showing that the headline result survives them; separating the confirmatory findings from the exploratory ones the data suggested along the way; and resisting the causal verb when the design only supports an associational one. None of this weakens a paper. A finding stated with its limits is more credible, more reproducible, and ages far better than one dressed up beyond what the evidence carries.

The checklist, in one place

• Pre-registration or a dated analysis plan, written before seeing results.
• A frozen, read-only analysis dataset with a recorded checksum.
• Code deposited publicly, archived with a DOI, and runnable end to end from a README.
• A pinned environment: exact dependency versions, fixed seeds, a container where it counts.
• A limitations section that states the finding’s real boundaries.

Build it into the workflow, not the deadline

What these five habits share is timing. Every one of them is nearly free when done while the work is live and expensive-to-impossible when reconstructed at submission. You cannot pre-register after you have seen the results; you cannot checksum a dataset you have already overwritten; you cannot pin an environment you have since upgraded past. Reproducibility is not a task you schedule for the week before submission — it is a set of small commitments the work either accumulated or did not.

This is the same logic that runs through everything we build. A manuscript in Dissertatio carries its own evidence: references resolved against Crossref and locked to real sources, named versions that mark exactly what the draft looked like at each milestone, and a record of every check that ran against it. The checklist above lives one layer down, in the data and the code, but the instinct is identical — capture the record as you go, so that when someone asks “can you reproduce this?” the answer is already yes, and provably so. Reproducibility, like credibility, is not something you assert at the end. It is something you leave a trail of from the beginning.