The learning study
Follow the calculation.
The starter package contains just learning-study.csv and study-sheet.md. Use it in a new folder while reading the tutorial on the site or from the separate reference package. The reference package is a completed NLSS project, including the tutorial, scripts, reports, original evidence and hidden .nlss/ files. Do not merge the two projects or run the learner prompts in the reference project if you want a clean comparison.
This is material 1.0: original teaching materials use CC BY 4.0, and original generator/verification scripts use Apache-2.0. See licenses and attribution. NLSS software retains its own existing license; no software runtime is bundled here. Installation and dependency setup are separate prerequisites.
How the teaching data were made
These details describe the fixed reference dataset; they are not instructions for the learner's analysis. The generator uses R's Mersenne-Twister RNG (Inversion / Rejection) and seed 20260918. After balanced random assignment, it draws one independent Normal(0, 0.5²) latent value per person. Scores follow Binomial(100, p), where logit(p) = logit(0.62) + 0.28 × I(practice) + u. Values were not clipped or redrawn, and the seed was not selected for statistical significance. The generator's logit difference is neither the observed score-point difference nor Cohen's d.
Three different checks
- Regenerate data: run the fixed generator with a new output filename. Compare its SHA-256 with the frozen CSV. The generator refuses replacement.
- Recalculate independently: use the checking script on an untouched reference project. It reads CSV first, calculates Welch formulas and calls base R directly, then compares the saved NLSS results, rows and plot layers.
- Replay NLSS: ask the agent to use the supported
replay-runutility on the specific saved request, in a copy of the complete reference project. Replay checks the recorded environment and evidence and creates a new run.
For an experienced R user, from the extracted reference folder:
Rscript scripts/generate-learning-study.R ../learning-study-regenerated.csv
Rscript scripts/check-reference.R . ../reference-check-new.json
The generator uses base R. The checking script also uses jsonlite, digest and arrow, all present in the validated environment. Run the independent checker before adding new analyses or replay runs: it deliberately requires exactly one original run of each reference module instead of guessing which analysis to validate. It refuses to overwrite its output file.
The original CSV SHA-256 is:
3dab41d156991ad77511263ebea54ed33946da2add84dfed95d9c8f6bca0fe72Ask for an exact replay
In this copy of the complete reference project, use NLSS replay-run for .nlss/runs/20260918T081427-247130f26b3/request.json. Keep the original evidence and current working data unchanged. Check the new completed result against the original. If environment or integrity verification fails, explain the specific mismatch without weakening the checks.Technical form for an agent with a resolved installation path:
Rscript "<installed-nlss>/scripts/R/run_nlss.R" replay-run \
--request "<copied-project>/.nlss/runs/20260918T081427-247130f26b3/request.json"
An isolated JSON file is insufficient; replay uses the project marker, preserved data, dictionaries, templates and artifacts. The run request lists R/package versions and other environment details. Exact replay can refuse a different environment even when a new scientific analysis would still be reasonable. It does not regenerate the agent's prose and needs no LLM inference itself.
Report integrity and presentation
The delivered reference-report.md was captured through project-report with its selected evidence, and its saved revision was explicitly integrity-checked. That is separate from the numerical and interpretive review. Keep hidden files when moving the project. material-manifest.json lists the reference files and SHA-256 hashes; SHA256SUMS can be checked with sha256sum -c SHA256SUMS. Both describe the original package, before any new runs.
One NLSS 2.0.1 presentation inconsistency is retained in the original protocol: its boxplot block says “Figure 1” while the image alt text says “Figure 2”. The protocol also uses the caption as its italic figure heading; the PNG has the requested title. Numerical results, plotted cases and image are unaffected. The authored reference page uses an explicit descriptive caption and alt text. Original NLSS evidence has not been edited to conceal this issue.