The Spacetime Metric
Level 6 · Research preparationDoctoral and independent-research pathwayAbout 30 hours

Open replication laboratory

Make apparatus, data, code, and disagreement portable enough for an independent lab to reproduce the result.

Design replication-ready protocols, calibration artifacts, sample and device provenance, blind interlaboratory studies, immutable data packages, executable environments, adversarial review, registered reports, and living meta-analysis for contested physical claims.

Measured physics

Before you begin

  • • Graduate experimental methods
  • • Statistics and scientific computing
  • • One topical Level 5 course

By the end, you can

  • • Convert an exploratory result into a replication protocol.
  • • Package hardware, samples, raw data, metadata, and code reproducibly.
  • • Run blinded independent replication and adjudicate discrepancies.
  • • Publish positive, null, and failed outcomes in a living evidence synthesis.

Interactive model

Explore before calculating

Independent laboratories connected through shared standards, blinded samples, immutable data, and an evidence ladder.
Replication is a designed transfer of a claim between people, apparatus, samples, and analysis environments—not a vague request to try again.

Live research workspace

Portable replication package

Mark only artifacts that another laboratory can actually inspect. The release gate stays closed until the claim, physical system, measurements, computation, and independent execution all have durable evidence.

Release stage

Define the claim

0/12 artifacts · 0%

Protocol
Apparatus & sample
Calibration
Data
Code
Independent review

Level 6 · Research preparation teaching kit

Record the investigation. Teach the reasoning.

A learner-facing lab record and a course-specific instructor guide turn the live model into a repeatable classroom investigation.

Learner record

Portable replication-package release gate

Can an independent team reproduce the apparatus state, raw-data lineage, analysis environment, and declared decision without private clarification?

Download learner record

Instructor guide

Teach for evidence, not button pushing

Researchers turn an exploratory result into a portable, independently executable evidence package that preserves positive, null, and failed outcomes.

Download instructor guide
Open the complete print-friendly teaching kit →

Advanced assessment

Reconstruct it. Quantify it. Try to break it.

Make artifacts portable, reruns independent, and disagreements diagnosable before declaring replication or failure. Three research-level challenges include explicit deliverables and scoring criteria.

1. derivation2. analysis3. design
Open assessment packLearner MarkdownInstructor Markdown

Portable research dataset

Record data that another laboratory can open.

Artifact integrity, clean-environment rerun, discrepancy, and release records. JSON preserves schema and provenance; CSV supports ordinary analysis tools. Imports stay in this browser and are limited to 1 MB and 5,000 records.

Download schemaDownload notebook

Ready for a new research record.

ArtifactlabelChecksumhashPresentbooleanIndependent resultlabelDiscrepancycategoryRelease statelabelRecord
Schema field definitions
Artifact · label
Required package artifact.
Checksum · hash
Integrity checksum or version.
Present · boolean
Artifact availability.
Independent result · label
Clean-environment rerun outcome.
Discrepancy · category
Protocol, apparatus, environment, data, analysis, or none.
Release state · label
Pass, hold, or remediation state.

Lesson 1 of 3

From exploratory effect to replication contract

Which details must remain fixed, and which must deliberately vary to test generality?

Direct replication preserves the core protocol, while conceptual replication changes implementation but tests the same prediction. Both need a frozen effect definition, sample size, exclusions, controls, and success threshold.

A registered report moves peer review before outcomes. Feasibility pilots should calibrate variance without consuming the confirmatory dataset or silently tuning the primary endpoint.

direct replicationconceptual replicationregistered reportprimary endpointpower analysis

Worked example

An exploratory effect was chosen after inspecting 20 outcomes. How should replication proceed?

  1. 1. Name one primary outcome before new data.
  2. 2. Estimate effect and uncertainty conservatively.
  3. 3. Power for a smaller plausible effect.
  4. 4. Treat other outcomes as secondary or exploratory.

The replication tests one frozen claim rather than repeating the original search process.

Try it

Protocol hardening workshop

Materials: Exploratory paper and lab notes

  1. 1. Identify researcher degrees of freedom.
  2. 2. Freeze primary protocol.
  3. 3. Add positive and negative controls.
  4. 4. Define deviations and abort rules.

Notice: Most irreproducibility risks become visible when another team tries to execute the protocol literally.

Check your understanding: Why can an exact procedural copy still be a weak replication?

Answer: It may reproduce the same hidden artifact, sample batch, or analysis dependency.

Conceptual and independent variants test generality and mechanism.

Lesson 2 of 3

Portable apparatus, samples, data, and code

Can another laboratory reconstruct what happened without private interpretation from the originators?

A package includes drawings, bills of materials, firmware, calibration certificates, sample provenance, environmental logs, raw immutable data, metadata schemas, and executable analysis environments.

Checksums, versioned containers, data dictionaries, and automated tests protect against silent drift. Sensitive or proprietary limits must be declared because they reduce independent inspectability.

provenancechecksumcontainerdata dictionarycomputational reproducibility

Worked example

Code runs only on the original laptop. What must change?

  1. 1. Freeze dependencies and runtime.
  2. 2. Add a minimal test dataset.
  3. 3. Document deterministic commands.
  4. 4. Verify from a clean machine.

A container or locked environment plus tests turns local execution into a portable artifact.

Try it

Clean-room reproduction

Materials: Draft package and fresh machine/account

  1. 1. Follow only published instructions.
  2. 2. Record every ambiguity.
  3. 3. Recreate headline table and figure.
  4. 4. Return failures as actionable package issues.

Notice: Clean-room failure is evidence about documentation and hidden state, not necessarily about the underlying result.

Check your understanding: Why are raw data and calibration data both needed?

Answer: Processed results cannot reveal whether calibration, filtering, or exclusions created the effect.

The full transformation chain must be inspectable.

Lesson 3 of 3

Discrepancy adjudication and living synthesis

When laboratories disagree, which experiment or analysis can resolve the difference?

Discrepancies may arise from sample state, apparatus sensitivity, environment, calibration, analysis, or chance. Blinded sample swaps, shared calibration artifacts, and cross-analysis of the same raw data localize the source.

A living meta-analysis includes all preregistered outcomes, models heterogeneity, and avoids treating publication count as independence. Disagreement becomes a research target rather than a reputational contest.

heterogeneitysample swapcross-analysismeta-analysispublication bias

Worked example

Lab A finds an effect, Lab B does not. What two exchanges best localize cause?

  1. 1. Have each lab analyze the same raw dataset.
  2. 2. Swap blinded samples or devices.
  3. 3. Use a shared calibration artifact.
  4. 4. Compare which result follows data, sample, or apparatus.

Cross-analysis and sample/device exchange separate analysis from physical implementation effects.

Try it

Discrepancy decision tree

Materials: Conflicting synthetic lab results

  1. 1. Test data-pipeline differences.
  2. 2. Test calibration and sensitivity.
  3. 3. Swap samples/devices.
  4. 4. Plan the smallest deciding experiment.

Notice: Many conflicts can be resolved more cheaply than repeating both studies at larger scale.

Check your understanding: Does a random-effects meta-analysis make biased inputs reliable?

Answer: No.

It models heterogeneity but cannot repair shared artifacts, selective reporting, or invalid measurement.

Formula-to-meaning deck

Read the equation in ordinary language.

N≈2(z_{1−α/2}+z_{1−β})²σ²/δ²

A two-group planning approximation links sample size to uncertainty, target effect, error rate, and power.

I²=max[0,(Q−df)/Q]

I-squared summarizes observed meta-analytic heterogeneity beyond sampling error.

SHA256(file)=digest

A cryptographic digest detects silent changes to data, code, and artifacts.

Independent practice

Problem set

Work each problem before opening its hint and solution.

  1. 1. Why power a replication for a smaller effect than the exploratory estimate?

    Reveal hint

    Consider winner's curse and selection.

    Reveal solution

    Exploratory estimates are often inflated; a conservative target reduces false failure and improves planning.

  2. 2. If two labs share one sample batch and analysis code, which dependencies remain correlated?

    Reveal hint

    Consider physical and computational sources.

    Reveal solution

    Sample-specific effects and shared analysis artifacts remain correlated despite different locations.

  3. 3. What does a changed SHA-256 digest establish?

    Reveal hint

    It compares bytes, not scientific meaning.

    Reveal solution

    The file bytes changed; it does not reveal whether the change was valid or what it means scientifically.

Derivation studio

Build the result, line by line.

Keep the assumptions visible so the mathematics remains auditable.

Starting point

Replication sample-size scaling

Standardized difference signal-to-noise grows as δ√N/σ

  1. 1. Set critical threshold for alpha.
  2. 2. Add power threshold for beta.
  3. 3. Solve for √N.
  4. 4. Square and include two-group allocation.

N∝σ²/δ² for fixed α and power

Halving the target effect requires roughly four times the sample size.

Starting point

Inverse-variance evidence synthesis

Independent estimates y_i with variances σ_i²

  1. 1. Write Gaussian log likelihood.
  2. 2. Differentiate with respect to common mean.
  3. 3. Set derivative to zero.
  4. 4. Solve for weighted mean and variance.

μ̂=Σ(y_i/σ_i²)/Σ(1/σ_i²)

Precise studies weigh more only when uncertainties are valid and independence assumptions hold.

Computational notebook

Turn the model into an experiment.

Interlaboratory replication coordinator

Which dependency explains disagreement, and what is the minimum deciding experiment?

Inputs

  • • Preregistered lab outcomes
  • • Raw data and calibration packages
  • • Sample/device provenance
  • • Protocol deviations

Algorithm

  1. 1. Reproduce each analysis cleanly.
  2. 2. Model lab and sample heterogeneity.
  3. 3. Simulate cross-analysis and swap outcomes.
  4. 4. Select the most informative next experiment.

Evidence to produce

  • • Reproduction status matrix
  • • Heterogeneity and dependency model
  • • Discrepancy-resolution protocol

Paper-reading studio

Interrogate the source, not its reputation.

Reconstruct the assumptions, reproduce one calculation, and stop at the boundary of the reported evidence.

Replication-readiness audit

Could an independent team reproduce the claim from the published package without privileged contact?

  1. 1. Freeze the exact claim and endpoint.
  2. 2. Audit apparatus/sample provenance.
  3. 3. Execute code from a clean environment.
  4. 4. Recompute power, uncertainty, and all-run synthesis.

Calculation to reproduce: Reproduce the headline estimate and one sensitivity or heterogeneity analysis from raw inputs.

Evidence boundary: Replication strengthens a bounded claim; it does not automatically establish the proposed mechanism or broader technological interpretation.

Graduate oral defense

Defend a bounded claim under pressure.

Argue the strongest support, state the strongest objection fairly, and identify evidence that could actually decide the issue.

Proposition

Null and discrepant replications are productive scientific outputs when the protocol and package are strong.

  1. 1. They exclude parameter regions and reveal hidden dependencies.
  2. 2. They correct inflated exploratory estimates.
  3. 3. They guide cheaper deciding experiments.

Strongest objection: Poor-fidelity replications can create false negatives and reputational conflict without testing the original effect fairly.

Deciding evidence: Protocol-fidelity audits, positive-control success, shared standards, and discrepancy experiments showing whether results follow sample, apparatus, or analysis.

Research practicum

Make the work inspectable before making it impressive.

Pre-register the decisive test, package every dependency, and pass explicit milestone gates before interpretation expands.

Publish an independent replication package

Does one contested claim reproduce under a protocol that another lab can execute and audit?

Preregister

  • • Freeze claim, endpoint, controls, and success threshold.
  • • Power conservatively and define deviations.
  • • Specify sample/device swaps and meta-analysis.

Reproducibility package

  • • Protocol, BOM, firmware, and calibration
  • • Sample/device provenance and hashes
  • • Raw data, metadata, and containerized code
  • • All-run manifest and reviewer responses

Milestone gates

  1. 1. Clean-room package execution
  2. 2. Positive-control and calibration pass
  3. 3. Blinded confirmatory run
  4. 4. Independent review and living synthesis update

Continue into the evidence

Source-linked next reading

Method and confidence tiers

The claim-specific evidence and replication contract.

Sources and evidence

The corpus required for independent reconstruction.

Chapter 1: Evidence ladder

Controls, falsifiers, replication, and confidence updates.