All examples

Power planning · Before collecting data

Plan for a difference that matters.

What would count as a meaningful effect—and how many participants would you need? Work through a power calculation from explicit assumptions.

The planning choices

We are planning a fictional study comparing two groups of different people, with equal numbers in each group and one final score per person. A difference of 5 points would matter. We expect a standard deviation of 12.5 points within each group and have no prior prediction about which group will score higher.

We want 80% power at a 5% significance level. These are planning assumptions for this exercise, not estimates from a previous study. No allowance for dropout has been specified.

How many participants would we need? No data have been collected.

Material 1.0 · Fictional planning assumptions · Mike Hammes · CC BY 4.0.

Try it with your agent

Plan before collecting data.

Give your agent the study sheet above, as a file or pasted text, then send this prompt. No dataset is needed.

Download planning brief (ZIP)

Assumptions, prompt and licenses · No dataset

Prompt for your NLSS agent
Use NLSS to help plan this study. How many participants would we need?

What would change if the scores varied more?

Consider your answer before opening the reference. The reference is one worked approach; compare the analysis choices before expecting identical numbers.

Compare with the reference plan

These saved results use the choices described below. Your agent may propose a different approach.

Plan for 100 analysable participants per group

Under the stated assumptions, the calculation requires 100 participants in each group: 200 in total. Rounding the continuous solution upward gives an attained power of .804, just above the target of .80.

Quantity Result
Meaningful difference / common SD 5 / 12.5
Standardized effect d = 0.40
Two-sided significance level .05
Target power .80
Continuous solution per group 99.0803
Rounded allocation 100 + 100
Attained power at that allocation .80365

This is parameter-only, a priori planning for an equal-variance two-sample t test. The NLSS calculation used the supplied effect and equal allocation; an independent pwr::pwr.t.test calculation confirmed the solution and rounded-sample power. The pwr documentation defines the inputs and sample-size unit.

Power .80 means that, under this assumed effect, design and model, repeated studies would reject the null approximately 80% of the time. It is not an 80% chance that an individual result is true, nor a guarantee of detecting a difference in the next study.

What the number depends on

The 5-point target and SD of 12.5 were stipulated for teaching; neither is supported by pilot evidence here. Smaller effects or greater variability would require more participants. The output is the number of analysable participants; it includes no allowance for attrition, clustering, multiple testing or unequal allocation. It is not an exact sample-size calculation for a Welch test or for the repeated-measures intervention example.

A real plan needs a defensible meaningful effect and variability assumption, plus design-specific choices about loss to follow-up and analysis. There is no dataset and no observed post-hoc power calculation in this example.

NLSS 2.0.1 · Material 1.0 · Mike Hammes · CC BY 4.0. See reproduction and evidence.