Choosing the Right Statistical Test
A decision framework for quantitative analysis, match your test to your variables, groups and assumptions, from t-tests to regression.
The most common statistics error in student research is not miscalculation, it is choosing the wrong test. Test selection follows logically from three things: the type of your variables, how many groups you compare, and whether your data meet the test's assumptions. Get those right and the test almost picks itself.
Step 1: Identify Your Variable Types
- Categorical (nominal): unordered groups, gender, treatment type.
- Ordinal: ordered but not evenly spaced, Likert responses.
- Continuous (interval/ratio): measured on a scale, age, blood pressure, score.
Your outcome and predictor variable types are the first fork in the decision tree.
Step 2: Comparing Groups on a Continuous Outcome
- Two independent groups: independent-samples t-test (or Mann-Whitney U if assumptions fail).
- Two related measurements: paired t-test (or Wilcoxon signed-rank).
- Three or more groups: one-way ANOVA (or Kruskal-Wallis), with post-hoc tests to locate differences.
Step 3: Relationships Between Variables
- Two continuous variables: Pearson correlation (or Spearman for ordinal/non-normal).
- Predicting a continuous outcome: linear regression (multiple regression for several predictors).
- Predicting a binary outcome: logistic regression.
- Two categorical variables: chi-square test of independence.
Step 4: Check the Assumptions
Parametric tests (t-test, ANOVA, Pearson, linear regression) assume roughly normal distributions and, often, equal variances. Check normality (Shapiro-Wilk, Q-Q plots) and homogeneity of variance (Levene's test). If assumptions fail and cannot be remedied, switch to the non-parametric equivalent, this is a judgement you must report, not hide.
Report the Choice, Not Just the Result
Markers want to see reasoning: "Because the outcome was continuous and normally distributed across two independent groups, an independent-samples t-test was used." That single sentence earns method marks the result alone does not.
Checklist
- Have I classified each variable's type correctly?
- Does the test match the number and relationship of groups?
- Have I checked (and reported) the assumptions?
- Did I justify the test choice in writing?
- Have I reported an effect size alongside the p-value?
- Are exact p-values given rather than bare inequality signs?
The Decision Table
Most student analyses resolve to one of the rows below. Find your outcome type on the left, then read across to what you are doing with it.
| Outcome | Comparison or relationship | Parametric test | Non-parametric alternative |
|---|---|---|---|
| Continuous | Two independent groups | Independent-samples t-test | Mann-Whitney U |
| Continuous | Two related measures | Paired t-test | Wilcoxon signed-rank |
| Continuous | Three or more independent groups | One-way ANOVA | Kruskal-Wallis |
| Continuous | Three or more related measures | Repeated-measures ANOVA | Friedman |
| Continuous | Association with another continuous variable | Pearson correlation | Spearman rho |
| Continuous | Prediction from one or more variables | Linear regression | Robust or quantile regression |
| Binary | Prediction from one or more variables | Logistic regression | Exact logistic regression for small samples |
| Categorical | Association with another categorical variable | Chi-square test of independence | Fisher's exact test |
| Count | Prediction from one or more variables | Poisson regression | Negative binomial where overdispersed |
Assumptions, and What Actually Happens When They Fail
| Test | Key assumptions | If violated |
|---|---|---|
| t-test | Normality of the outcome, equal variances, independent observations | Use Welch's t-test for unequal variances, Mann-Whitney for marked non-normality |
| ANOVA | Normality, homogeneity of variance, independence | Welch's ANOVA, or Kruskal-Wallis |
| Pearson correlation | Linear relationship, roughly bivariate normal, no strong outliers | Spearman rho on ranks |
| Linear regression | Linearity, independent errors, constant variance, normal residuals | Transform the outcome, or use robust standard errors |
| Chi-square | Expected count of at least five in most cells | Fisher's exact test |
| Logistic regression | Independence, no perfect separation, adequate events per predictor | Reduce predictors, or use penalised estimation |
One caution about normality testing. Shapiro-Wilk becomes very sensitive in large samples, flagging trivial departures as significant, while in small samples it can miss real ones. Look at a Q-Q plot and a histogram alongside the test rather than treating the p-value as the verdict.
Report an Effect Size, Not Only a p-Value
A p-value tells you how surprising the data would be if there were no effect. It does not tell you how large the effect is. Most marking schemes now expect both, and APA style requires an effect size wherever one is available.
| Test | Usual effect size |
|---|---|
| t-test | Cohen's d |
| ANOVA | Eta squared or partial eta squared |
| Correlation | r itself, and r squared |
| Linear regression | R squared, and standardised coefficients |
| Logistic regression | Odds ratio with a confidence interval |
| Chi-square | Cramer's V, or phi for a two-by-two table |
Three Worked Decisions
Reading the table is easier than applying it, so here are three ordinary student scenarios worked through.
Scenario one. You surveyed 140 undergraduates and want to know whether wellbeing scores differ between students living at home and students in halls. The outcome is a summed wellbeing scale, so continuous. There are two groups, and they are independent, since nobody is in both. That points to an independent-samples t-test. You check the distributions, find them acceptably symmetric, but Levene's test flags unequal variances, so you use Welch's version and report Cohen's d alongside the p-value.
Scenario two. You measured anxiety in the same 40 participants before and after an eight-week programme. The outcome is continuous, but the two measurements come from the same people, so they are not independent. That is a paired t-test. The paired structure matters enormously here, because treating the measurements as independent throws away the very information that gives the design its power, and usually understates the effect.
Scenario three. You want to know whether course choice is associated with whether students used the library's study-skills service. Both variables are categorical, so a chi-square test of independence fits. One cell has an expected count of three, below the usual threshold, so you switch to Fisher's exact test and report Cramer's V as the effect size. Note that you decided this from the expected counts, not the observed ones.
In all three, the reasoning came before the software. That order is what a marker is looking for.
Common Mistakes and Fixes
| Mistake | Fix |
|---|---|
| Multiple t-tests instead of one ANOVA | Run the ANOVA, then post-hoc tests with a correction |
| Treating a Likert item as continuous without comment | Justify it, or use ordinal methods, and say which you chose |
| Ignoring paired structure in the data | Use the paired or repeated-measures version of the test |
| Reading correlation as causation | Describe association, and reserve causal language for designs that support it |
| Reporting "p < .05" only | Give the exact value, the test statistic, degrees of freedom and an effect size |
| Running every test and reporting the significant one | Pre-specify the analysis, and disclose exploratory tests as exploratory |
Discipline Notes
- Psychology: APA reporting conventions apply, and effect sizes with confidence intervals are expected.
- Health and medicine: confidence intervals often matter more than p-values, and odds or risk ratios are the common currency.
- Business and economics: regression dominates, with attention to model diagnostics and robustness checks.
- Education and social sciences: ordinal and clustered data are common, so justify how you handled scale type and nesting.
Where Ethical Support Fits
Most universities run a statistics advisory service, and using it is expected rather than exceptional. Asking whether your design calls for a repeated-measures model, or how to read a diagnostic plot, is ordinary research training.
What stays yours is running the analysis, reading the output and writing the interpretation. If a result is not significant, that is your result. Reporting it plainly is better research than searching for a test that produces a more agreeable number.
Frequently Asked Questions
Can I treat Likert data as continuous?
Single items are ordinal. Summed multi-item scales are commonly treated as continuous, and that is widely accepted where the scale has several items and behaves reasonably. Whichever you choose, state the choice and the reason.
What if my data are not normally distributed?
Check how far from normal, and how large your sample is. Parametric tests are fairly robust in larger samples. If the departure is severe, use the non-parametric equivalent and report why.
Do I need to correct for multiple comparisons?
If you are running many tests on the same data, yes. Bonferroni is the simplest and the most conservative. Whatever you use, name it.
How big does my sample need to be?
That comes from a power calculation using your expected effect size, alpha and desired power, not from a round number. Run it before collecting data if you can.
My result is not significant. Have I failed?
No. A well-designed study with a null result is a legitimate contribution. What weakens a dissertation is not the null result but a failure to discuss it, including whether the study was powered to detect the effect at all.
Which software should I use?
Whichever your department teaches and supports. SPSS, R, Stata and Python will all run these tests. Marks come from justification and interpretation, not from the software name.
Your Next Step Today
Write one sentence for each analysis in your results chapter, in this form: because the outcome was X and I compared Y, I used Z, having checked assumption A. If you cannot complete the sentence, the test choice is not yet defensible, and that is the thing to fix before you run anything else.
Trusted Sources
- UCLA Office of Advanced Research Computing, Choosing the Correct Statistical Test, with worked examples in SPSS, Stata, SAS and R. Accessed 11 August 2026.
- BMJ, Statistics at Square One, free chapter-by-chapter introduction. Accessed 11 August 2026.
- American Psychological Association, Numbers and Statistics Guide (PDF). Accessed 11 August 2026.
- University of Southern California, Quantitative Methods. Accessed 11 August 2026.
Conventions differ by discipline and by supervisor. Where this guide and your department's guidance differ, follow your department.
Related services
Related guides
- Reporting Statistics in APA Style: Numbers, Tables and p-Values
- Reliability and Validity in Research
- Writing the Methodology Chapter: Design, Sampling and Justification
- Medical Research: Data Collection, Analysis and Reporting
All academic writing guides or browse our academic writing services.