A guided comparison · two conditions
More learners.
More words.
What makes a difference?
Start with an example, change the study size, and see which expansion reduces variation in the average effect.
Restarting loads the example design, scoring example, counts, variation and 100% increase, and clears added predictors and simulation settings. Kept assumption sets remain and use the reset counts. Resume keeps your current inputs.
Illustrative assumptions, not a sample-size recommendation or a power analysis.
- 01
Set your starting point
Choose learner and word counts. Review the assumptions.
- 02
Explore the difference
Move along the curves and compare actual study sizes.
- 03
Take it into R
Download the inputs, calculations, and a reproducible figure.
Inferential target
Start with the claim, not the coefficient
Choose one primary question. If the paper makes several claims, run each through its own branch; one coefficient does not answer all four.
Who scored higher?
Score reliability for individual differences
Evidence and reporting guidance for this question
Aligned evidence
Minimum reporting
This does not license
This branch can use a closed-form sensitivity calculation after the score unit is defined.
Closed-form sensitivity
Expected alpha/KR-20
Prediction from typed assumptions, not an observed coefficient and never a pass/fail threshold. These inputs are separate from the scoring illustration. Do not add constant or different-purpose fillers to k while holding the average correlation fixed.
Step 1 of 3 · Your starting point
What does your study look like?
This example compares two conditions. Each learner sees words in both conditions; each word appears in both conditions across learners. Different design? Review the structure. Including known words? Define the scores.
Review or change the variation assumptions Start with example values
These values describe how much the difference between the two conditions varies. Larger values mean more variation. The starting values are examples for exploration, not estimates or recommended settings for your study.
Checking the box removes the memory component from this calculation. It assumes that component is fully accounted for; simply measuring memory does not establish this. Uncertainty in estimating the moderator effect is not included.
How do I choose values if I am unsure?
- Use comparable evidence when available. Look for slope SDs from a pilot or prior study with a comparable task, the same outcome model and the same condition coding. These inputs use the latent logit scale; raw-score SDs, accuracy percentages and standard errors are not substitutes.
- Explore uncertainty. Without suitable estimates, use the example to learn the tool. Then try lower and higher plausible values, changing one assumption at a time. There is no universal cutoff for a small or large value.
- Check whether the conclusion changes. Compare the same study sizes again. If the preferred expansion changes, the choice depends on the assumptions; seek better evidence before committing. Record the values you tried and why.
These hypothetical counts are separate from the structural audit's recorded counts.
Step 2 of 3 · Explore the difference
Where would extra effort help?
Lower curves mean the average effect would vary less if you repeated the study with new learners and words.
Hover to inspect. Click or use the slider to select. Arrow keys move the slider. Counts round up to whole learners or words per condition.
A smaller sampling SD means less variation across repeated samples. It does not tell you the chance of detecting an effect (power).
Keep up to three sets to compare how the conclusion changes.
The two expansions need not cost the same. Learning time and the response process are held constant here; extra words may change exposure, fatigue, or difficulty.
Exact values and variance contributions
| Plan | Learners / total words | Projected SD | SD reduction |
|---|
- Main bottleneck
- —
- Participant variance term
- —
- Item variance term
- —
Compare assumption sets 0 of 3 kept
Keep a set, change the variation assumptions, then keep another. Each set uses the same study sizes and selected increase; changing either recalculates all sets.
Only the sets you keep are compared here. Agreement does not establish robustness beyond those sets or estimate power. Sets stay on this page until reload; the R download includes their inputs and comparison.
This projection excludes response-level estimation error. It is not a fitted GLMM standard error, power, or a validation of your study.
Step 3 of 3 · Keep the reasoning
Reproduce this comparison in R.
Download a standalone script with your current inputs, selected comparison, and sensitivity curves. It uses base R; no packages or uploads are needed.
Inspect the R code before downloading
Before choosing a design
The script reproduces the closed-form projection. It does not simulate responses, fit a model, or estimate power.
Published simulation snapshot
Exact reference-grid lookup
The selected row is copied from a published 500-replication grid. No interpolation or extrapolation is performed, and the row is never applied automatically to the current design.
Qualitative route
No single-number answer
Use the aligned evidence and reporting prompts above.
Drafting prompt
Reporting starter
Inspect or change the design structure
Review an R simulation result
After running the downloaded R program, choose summary.csv from its output folder. The file is read only on this device. Start with the fit counts, then interpret rejection and coverage; results stay separate from your current planner settings.
No file selected.
1. Check fit quality
2. Read rates with their denominators
Rejection bounds for missing results
How to interpret these results
Wilson intervals describe Monte Carlo uncertainty in the displayed rates; they are not intervals for the model coefficient. Coverage measures how often the model's 95% coefficient interval contains the generating coefficient. Monte Carlo SE can be zero when every usable replication has the same outcome; the Wilson interval still reflects finite simulation size.
Excluding singular fits changes the subset being summarized. Neither subset corrects bias caused by unsuccessful fits. Inspect individual warnings, estimates, eligibility and feedback scores in replications.csv, and the generating assumptions in assumptions.R. This summary does not identify the full design or model.
Run the tested coefficient at zero to examine Type I error under the same nuisance effects. Compare plausible assumptions before making a design decision. Independent known-word screening and feedback-only fillers do not model a motivation effect.
What this planner can and cannot tell you
Explore assumptions and identify structural problems. The planner cannot certify a design or replace a fitted analysis. The numerical validation registry is empty; passing a structural check does not establish numerical performance.