How is repeating an analysis different from repeating a study?

Two teams can check the same scientific claim in different ways. One can run the original data through the original analysis to see whether the reported result follows. Another can collect new data addressing the same question. The first checks a computational path; the second checks whether a finding recurs in a new body of evidence. Both are valuable, but success at one does not automatically establish success at the other.

The U.S. National Academies uses reproducibility for consistent results from the same input data and computational procedures, and replicability for consistency across studies with their own data. Terminology varies across fields, so an explanation should state which meaning it uses. The distinction is practical: original data and code can help another analyst reproduce a calculation, while replication requires another study with suitable methods and observations.

Imagine a fictional experiment measuring how a coating affects cooling. The original team records temperatures and calculates an average difference. A second analyst runs the shared code on those same records. If the published average cannot be recovered, a data-processing or reporting problem may exist. If it can be recovered, that establishes something about the calculation. It does not show that another set of coated samples will behave the same way.

A new experiment supplies that different kind of check. Its result need not match every decimal place because samples and measurements vary. The question is whether the findings are consistent given their uncertainty and the intended conditions. If they differ, investigators need to examine possible explanations: changed materials, changed procedures, random variation, an overlooked factor or an error. A disagreement is evidence to investigate, not an automatic verdict that one team acted dishonestly.

Generalization asks a further question. A result may recur with the same material in a laboratory yet not apply to every outdoor building. Testing a broader setting changes what the evidence can support. It helps to keep three claims separate: the reported calculation can be recovered; the result recurs in another suitable study; and the result applies across additional contexts. Each requires evidence appropriate to its scope, and none is established merely by repeating the same sentence in many articles.

The diagram places the original data on one branch and new observations on another. That visual separation is the key. Transparent records make a calculation checkable; independent observations test whether the finding extends beyond the original sample. Scientific confidence can grow when these different checks agree, while disagreement can reveal a boundary or a mistake worth understanding. Knowledge becomes more dependable through these opportunities for correction, not through the assumption that one successful study has ended the inquiry.