A guided comparison · two conditions
More learners.
More words.
What makes a difference?
Start with an example, change the study size, and see which expansion reduces variation in the average effect.
Restarting loads the example design, counts, variation and 100% increase. Kept assumption sets remain and use the reset counts. Resume keeps your current inputs.
Illustrative assumptions, not a sample-size recommendation or a power analysis.
- 01
Set your starting point
Choose learner and word counts. Review the assumptions.
- 02
Explore the difference
Move along the curves and compare actual study sizes.
- 03
Take it into R
Download the inputs, calculations, and a reproducible figure.
Inferential target
Start with the claim, not the coefficient
Choose one primary question. If the paper makes several claims, run each through its own branch; one coefficient does not answer all four.
Who scored higher?
Score reliability for individual differences
Evidence and reporting guidance for this question
Aligned evidence
Minimum reporting
This does not license
This branch can use a closed-form sensitivity calculation after the score unit is defined.
Closed-form sensitivity
Expected alpha/KR-20
Prediction from typed assumptions, not an observed coefficient and never a pass/fail threshold.
Step 1 of 3 · Your starting point
What does your study look like?
This example compares two conditions. Each learner sees words in both conditions; each word appears in both conditions across learners. Different design? Review the structure.
Review or change the variation assumptions Start with example values
These values describe how much the difference between the two conditions varies. Larger values mean more variation. The starting values are examples for exploration, not estimates or recommended settings for your study.
Checking the box removes the memory component from this calculation. It assumes that component is fully accounted for; simply measuring memory does not establish this. Uncertainty in estimating the moderator effect is not included.
How do I choose values if I am unsure?
- Use comparable evidence when available. Look for slope SDs from a pilot or prior study with a comparable task, the same outcome model and the same condition coding. These inputs use the latent logit scale; raw-score SDs, accuracy percentages and standard errors are not substitutes.
- Explore uncertainty. Without suitable estimates, use the example to learn the tool. Then try lower and higher plausible values, changing one assumption at a time. There is no universal cutoff for a small or large value.
- Check whether the conclusion changes. Compare the same study sizes again. If the preferred expansion changes, the choice depends on the assumptions; seek better evidence before committing. Record the values you tried and why.
These hypothetical counts are separate from the structural audit's recorded counts.
Step 2 of 3 · Explore the difference
Where would extra effort help?
Lower curves mean the average effect would vary less if you repeated the study with new learners and words.
Hover to inspect. Click or use the slider to select. Arrow keys move the slider. Counts round up to whole learners or words per condition.
A smaller sampling SD means less variation across repeated samples. It does not tell you the chance of detecting an effect (power).
Keep up to three sets to compare how the conclusion changes.
The two expansions need not cost the same. Learning time and the response process are held constant here; extra words may change exposure, fatigue, or difficulty.
Exact values and variance contributions
| Plan | Learners / total words | Projected SD | SD reduction |
|---|
- Main bottleneck
- —
- Participant variance term
- —
- Item variance term
- —
Compare assumption sets 0 of 3 kept
Keep a set, change the variation assumptions, then keep another. Each set uses the same study sizes and selected increase; changing either recalculates all sets.
Only the sets you keep are compared here. Agreement does not establish robustness beyond those sets or estimate power. Sets stay on this page until reload; the R download includes their inputs and comparison.
This projection excludes response-level estimation error. It is not a fitted GLMM standard error, power, or a validation of your study.
Step 3 of 3 · Keep the reasoning
Reproduce this comparison in R.
Download a standalone script with your current inputs, selected comparison, and sensitivity curves. It uses base R; no packages or uploads are needed.
Inspect the R code before downloading
Before choosing a design
The script reproduces the closed-form projection. It does not simulate responses, fit a model, or estimate power.
Published simulation snapshot
Exact reference-grid lookup
The selected row is copied from a published 500-replication grid. No interpolation or extrapolation is performed, and the row is never applied automatically to the current design.
Qualitative route
No single-number answer
Use the aligned evidence and reporting prompts above.
Drafting prompt
Reporting starter
Inspect or change the design structure
What this planner can and cannot tell you
Explore assumptions and identify structural problems. The planner cannot certify a design or replace a fitted analysis. The numerical validation registry is empty; passing a structural check does not establish numerical performance.