Guide

How to choose a statistical test

Start with the question, then identify the variables and how observations are arranged. This guide explains why common tests may fit, what to check, and when the available information is not enough to choose uniquely.

Updated September 28, 2026Research stage: Collect and analyseNot yet checked by a named reviewer

Start with the question, not the test name

A statistical test is a method for answering a particular kind of question with a particular arrangement of data. Begin by writing what you want to learn in one sentence. Decide whether you are describing a sample, comparing groups or measurements, examining an association, or predicting an outcome. Then identify the variables, their measurement levels, and whether observations are independent or paired. This order follows standard research-methods guidance (Field, 2018; Howell, 2013).

State the research question and the population or sample it concerns.
Name the outcome, predictor, grouping factor, covariate, or comparison value. A relationship question may have two variables without designating either as the outcome.
Record whether each variable is categorical, ordinal, or quantitative, and how many categories or groups it has.
Establish whether the observations come from separate people, repeated measurements of the same people, or matched pairs.
Only then compare candidate methods and check their assumptions against the data and study design.

Choosing a statistical test: a decision treeFollow the questions from the top. Each path ends in the tests commonly used for that situation. It shows common practice, not the only valid choice.

What are you trying to find out?

Describe a variable
How is the variable measured?
- Nominal (categories)
  Commonly used: Mode, Frequency
  May also fit: Percentage
- Ordinal (ordered categories)
  Commonly used: Median
  May also fit: Frequency, Mode
- Interval or ratio, roughly symmetric
  Commonly used: Mean, Standard deviation
  May also fit: Median
- Interval or ratio, skewed
  Commonly used: Median
  May also fit: Mean, Standard deviation
Compare groups, times or a target value
What are you comparing?
- One group with a stated value
  How is the outcome measured?
  - Interval or ratio
    Commonly used: One-sample t-test
  - Nominal (categories)
    Commonly used: Chi-square goodness-of-fit test
- Separate groups of different people (independent)
  How is the outcome measured?
  - Interval or ratio
    How are the groups arranged?
    Two groups, one grouping variable
    Commonly used: Independent-samples t-test
    Three or more groups, one grouping variable
    Commonly used: One-way ANOVA
    Two grouping variables together
    Commonly used: Two-way ANOVA
    May also fit: Independent-samples t-test, Multiple regression
    Each grouping variable can have two or more categories.
    Groups compared while controlling for a numeric covariate
    Commonly used: ANCOVA
    May also fit: Independent-samples t-test
  - Ordinal
    How many groups?
    Two
    Commonly used: Mann–Whitney U test
    Three or more
    Commonly used: Kruskal–Wallis test
  - Nominal (categories)
    Commonly used: Chi-square test of independence
    May also fit: Fisher's exact test
    The same test applies with more groups or categories.
- The same people measured more than once (paired)
  How is the outcome measured, and how many times?
  - Interval or ratio, twice
    Commonly used: Paired-samples t-test
  - Interval or ratio, three or more times
    Commonly used: Repeated measures ANOVA
  - Ordinal, twice
    Commonly used: Wilcoxon signed-rank test
Examine a relationship between two variables
How are the two variables measured?
- Both interval or ratio, roughly normal
  Commonly used: Pearson correlation
  May also fit: Simple linear regression
- At least one ordinal
  Commonly used: Spearman's rank correlation
Predict or explain an outcome
How is the outcome measured, and how many predictors?
- Interval or ratio, one quantitative predictor
  Commonly used: Simple linear regression
  May also fit: Pearson correlation
- Interval or ratio, two or more predictors
  Commonly used: Multiple regression
- Two categories, such as yes or no
  Commonly used: Logistic regression
Examine categories
How many categorical variables?
- One, compared with expected shares
  Commonly used: Chi-square goodness-of-fit test
- Two, examined together
  Commonly used: Chi-square test of independence
  May also fit: Fisher's exact test

Separate comparison, association and prediction

The wording of a question helps distinguish the analysis family. A comparison asks whether an outcome differs between groups, measurements, or a value set in advance. An association asks whether variables vary together without claiming that one predicts or causes the other. Prediction estimates an outcome from one or more predictors; a prediction model does not by itself establish causation.

Three common purposes and the structure each asks about Scroll sideways to see every column.
Purpose	Question form	Variable roles	Typical starting point
Compare	Do groups or measurements differ?	Outcome plus grouping factor, time/condition, or stated value	Measurement level and independent versus paired observations
Association	Are these variables related?	Two variables considered together; neither must be an outcome	Variable types and whether the data are paired within a person
Predict	How well does one or more variables estimate an outcome?	Outcome plus one or more predictors	Outcome type, predictor types, and model assumptions

In a management study, for example, comparing satisfaction by bank type treats bank type as the grouping variable and satisfaction as the outcome. Relating study hours to exam scores treats the two quantitative measurements symmetrically. Predicting exam score from study hours assigns study hours as predictor and exam score as outcome. The design and the research question, not the names alone, determine those roles.

Grouping variable (independent variable)Bank type

Public banks
Private banks

Outcome (dependent variable)Job satisfaction

Two independent groups compared on one outcomeA grouping variable, bank type, splits participants into two separate groups, public and private; their mean job satisfaction is compared.

First variable (quantitative)Study time

Second variable (quantitative)Exam score

Two quantitative variables examined togetherStudy time and exam score are examined together, with no variable treated as the outcome.

Predictor (independent variable)Study hours

Outcome (dependent variable)Exam score

One predictor used to estimate an outcomeStudy hours, the predictor, points to exam score, the outcome being predicted.

Predictor 1Study hours

Predictor 2Attendance

Outcome (dependent variable)Exam score

Several predictors used together to estimate an outcomeTwo predictors, study hours and attendance, both point to exam score. Each one's contribution is estimated while holding the other constant.

First variable (categorical)Gender

Second variable (categorical)Employment status

Two categorical variables examined togetherGender and employment status, both categorical, are cross-tabulated to see whether they are associated.

Match summaries to the measurement level

Before testing a hypothesis, describe the observations. Frequencies and percentages summarize categories. The mode identifies the most frequent value and can describe nominal data. The median is meaningful when values can be ordered; the mean and standard deviation summarize quantitative data when an average and spread around it are informative. A skewed distribution or influential extremes can make the mean and standard deviation misleading, so inspect the data and consider the median and interquartile range as well.

Days of training attended by nine employees (invented for illustration) Scroll sideways to see every column.
Employee	Days
1	2
2	3
3	3
4	3
5	4
6	5
7	5
8	6
9	14

Summaries of the training days Scroll sideways to see every column.
Statistic	Value	Working	What it tells you
Mean	5.0	(2 + 3 + 3 + 3 + 4 + 5 + 5 + 6 + 14) ÷ 9 = 45 ÷ 9	The average, pulled upwards by the one employee with 14 days.
Median	4.0	The 5th of the 9 values in order	The middle employee attended 4 days: half attended fewer or the same, half the same or more.
Mode	3	3 appears three times, more than any other value	The most common number of days.
Standard deviation	3.61	√[ Σ(x − x̄)² ÷ (n − 1) ]	Values typically lie a few days from the mean; the one large value inflates it.

The calculations in the example are deterministic summaries of a small invented dataset, not inferential evidence. A mean or median describes the observed sample; it does not on its own test a hypothesis or justify generalizing to a wider population.

Independent groups are not paired measurements

Independent observations come from distinct units: each participant contributes to one group only. Paired observations are linked, most often because the same participant is measured twice or because participants are deliberately matched. The analysis must preserve that link. Treating paired scores as independent discards useful within-person information; treating independent groups as paired invents a link that is not in the design.

Independent groups

Group A

Participant 1
Participant 2
Participant 3

Group B

Participant 4
Participant 5
Participant 6

Six different people: each appears in one group only, so the two sets of scores are unrelated.

Paired observations

Participant 1 measured Before then After
Participant 2 measured Before then After
Participant 3 measured Before then After

Three people, each measured twice: every “after” score belongs with one “before” score.

Independent groups and paired observationsPaired scores are related: a person who scores high before usually scores high after. Paired tests use that link by analysing each person's change, which removes differences between people from the comparison. Treating paired data as independent ignores the link; treating independent groups as paired invents one. Either way the test answers the wrong question.

For a paired t-test, calculate each person's change and analyse those differences. Its normality assumption concerns the distribution of differences, not each time point separately. With independent groups, use a method for separate groups and check the outcome distribution within each group. The figure and the candidate profiles show how these structures lead to different methods.

Outcome (quantitative)Weekly working hours

Comparison value (stated in advance)40 hours

One sample compared with a stated valueThe mean of one quantitative variable, weekly working hours, is compared with a value stated in advance: the 40 hours in the contract.

Grouping variable (independent variable)Bank type

Public banks
Private banks

Outcome (dependent variable)Job satisfaction

Two independent groups compared on one outcomeA grouping variable, bank type, splits participants into two separate groups, public and private; their mean job satisfaction is compared.

First measurementScore before training

Second measurement, same participantsScore after training

The same participants measured twiceEach participant's score before training is compared with their own score after training.

Grouping variable (factor)Teaching method

Lecture
Discussion
Online

Outcome (dependent variable)Exam score

Three or more independent groups compared on one outcomeA grouping variable, teaching method, forms three separate groups: lecture, discussion and online. Their mean exam scores are compared.

Compare common tests by their data structure

The names can sound similar while answering different questions. A one-sample t-test compares one quantitative sample with a value specified in advance. An independent-samples t-test compares two separate groups; a paired-samples t-test compares linked measurements. One-way ANOVA compares three or more independent group means. Two-way ANOVA includes two grouping factors and their interaction. ANCOVA compares group means while adjusting for quantitative covariates. Chi-square tests use counts: the test of independence examines two categorical variables, while goodness-of-fit compares one categorical variable with expected proportions (Field, 2018; Howell, 2013).

Common statistical tests compared Scroll sideways to see every column.
Common candidate test	Research purpose	Outcome	Groups or variables	Structure	Key assumptions	Alternative or related method
One-sample t-test	Does the mean of a quantitative outcome differ from a known or hypothesised value?	One quantitative variable (interval or ratio).	None: a comparison value stated in advance. Groups: One.	Not applicable: one measurement per participant.	Interval or ratio measurement Independence of observations Normality No influential outliers	One-sample Wilcoxon signed-rank test
Independent-samples t-test	Do two separate groups differ in the mean of a quantitative outcome?	One quantitative variable (interval or ratio).	One grouping variable with two categories. Groups: Two.	Independent: each participant is in one group only.	Interval or ratio measurement Independence of observations Normality Homogeneity of variance No influential outliers	Mann–Whitney U test
Paired-samples t-test	Does the mean of a quantitative outcome change between two measurements of the same people?	One quantitative variable, measured twice.	Time or condition: two measurements. Groups: One group, two measurements.	Paired: each participant contributes both measurements, or participants are matched in pairs.	Interval or ratio measurement Paired observations Normally distributed differences No influential outliers	Wilcoxon signed-rank test
One-way ANOVA	Do three or more independent groups differ in the mean of a quantitative outcome?	One quantitative variable (interval or ratio).	One grouping variable (factor) with three or more categories. Groups: Three or more.	Independent: each participant is in one group only.	Interval or ratio measurement Independence of observations Normality Homogeneity of variance No influential outliers Adequate group sizes	Kruskal–Wallis test
Two-way ANOVA	Do two grouping variables, separately and together, relate to differences in a quantitative outcome?	One quantitative variable (interval or ratio).	Two grouping variables (factors), each with two or more categories. Groups: Every combination of the two factors' categories.	Independent: each participant is in one combination only.	Interval or ratio measurement Independence of observations Normality Homogeneity of variance No influential outliers Adequate group sizes	Aligned rank transform ANOVA
ANCOVA	Do groups differ in the mean of a quantitative outcome once a quantitative covariate is allowed for?	One quantitative variable (interval or ratio).	One or more grouping variables, plus one or more quantitative covariates. Groups: Two or more.	Independent: each participant is in one group only.	Interval or ratio measurement Independence of observations Normally distributed residuals Homogeneity of variance Linearity Homogeneity of regression slopes Covariate independent of the grouping No influential outliers	Quade's rank analysis of covariance
Chi-square test of independence	Are two categorical variables associated?	One categorical variable.	A second categorical variable. Groups: Two or more categories of each.	Independent: each participant is counted once, in one cell.	Categorical variables Independence of observations Adequate expected counts	Fisher's exact test
Chi-square goodness-of-fit test	Do the counts in one categorical variable's categories match shares stated in advance?	One categorical variable.	None: expected proportions stated in advance. Groups: Two or more categories.	Independent: each participant falls in one category.	Independent observations. Each participant falls in exactly one category. Expected counts of at least 5 in most categories.	Chi-square test of independence
Pearson correlation	Do two quantitative variables go together in a straight line, and how strongly?	First variable: quantitative.	Second variable: quantitative. Neither needs to be the outcome. Groups: Not applicable: one group, a pair of values per participant.	Each participant contributes one value of each variable.	Interval or ratio measurement Independence of observations Linearity Normality No influential outliers	Spearman's rank correlation
Simple linear regression	How well can a quantitative outcome be predicted from one predictor, and how much does it change per unit of the predictor?	One quantitative variable.	One predictor, usually quantitative. Groups: Not applicable.	Each participant contributes one value of each variable.	Interval or ratio measurement Independence of observations Linearity Normally distributed residuals Homoscedasticity Independent errors No influential outliers	Spearman's rank correlation
Multiple regression	How well do several predictors together predict a quantitative outcome, and what does each add while the others are held constant?	One quantitative variable.	Two or more predictors, quantitative or categorical (entered as dummy variables). Groups: Not applicable.	Each participant contributes one value of each variable.	Interval or ratio measurement Independence of observations Linearity Normally distributed residuals Homoscedasticity Independent errors No influential outliers No multicollinearity	Robust (heteroscedasticity-consistent) standard errors

The table scrolls sideways on narrow screens.

The matrix is a starting comparison, not a substitute for the design. For example, ANOVA's overall result indicates that not all group means are equal, but does not identify which groups differ; planned contrasts or appropriate post-hoc comparisons address that next question. A two-way ANOVA also tests whether one factor's relationship with the outcome changes across the other factor, called an interaction.

Factor ATraining method

Factor BExperience level

Outcome (dependent variable)Sales performance

Interaction: does the effect of training method depend on experience level?

Two grouping variables and one outcomeTwo grouping variables, training method and experience level, both point to one outcome, sales performance. The analysis also asks whether the effect of one depends on the other.

Grouping variable (factor)Teaching method

Covariate (quantitative)Prior achievement

Outcome (dependent variable)Final score

Groups compared while adjusting for a covariateA grouping variable, teaching method, and a quantitative covariate, prior achievement, both point to the outcome, final score. Groups are compared after allowing for prior achievement.

Categorical variable (observed counts)Service channel chosen

Expected counts (stated in advance)Equal shares for every channel

One categorical variable compared with expected sharesThe counts of customers choosing each service channel are compared with the counts expected if every channel were chosen equally often.

Test profiles: hypotheses, assumptions and reporting

A candidate is not a verdict. Each profile states the question it addresses, the roles and structure it needs, its null and alternative hypotheses, why it may fit, common confusions, assumptions, limitations, alternatives, and a reporting template. The Statistical Assumption Checker gives a fuller account of how to examine assumptions where it has a guide. Assumptions should be checked against the design and data, not treated as boxes to tick mechanically.

One-sample t-test

Compares the mean of a numeric outcome in one sample with a known or hypothesised value, such as a target or a published average.

What question it answers

Does the mean of a quantitative outcome differ from a known or hypothesised value?

Data it needs

Outcome: One quantitative variable (interval or ratio).
Predictor or grouping variable: None: a comparison value stated in advance.
Groups: One.
Independent or paired: Not applicable: one measurement per participant.

Outcome (quantitative)Weekly working hours

Comparison value (stated in advance)40 hours

One sample compared with a stated valueThe mean of one quantitative variable, weekly working hours, is compared with a value stated in advance: the 40 hours in the contract.

Why it may fit

It compares one sample's mean with a value set before the data were collected.

Why a similar test may not fit

Independent-samples t-test: The independent-samples t-test needs a second group of participants; here the comparison is with a fixed value.

Hypotheses

Null hypothesis (H₀): The population mean equals the stated value (μ = μ₀).
Alternative hypothesis (H₁): The population mean differs from the stated value (μ ≠ μ₀), or, if stated in advance, is larger or smaller.

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Normality: The outcome is approximately normally distributed within each group.
No influential outliers: No single case distorts the result.

Example question

Is the mean weekly working time of bank tellers different from the 40 hours set in their contracts?

Limitations

The comparison value must come from outside the sample, and its source should be stated.
Sensitive to outliers in small samples.

Alternatives

One-sample Wilcoxon signed-rank test: When the outcome is ordinal, or not normal in a small sample; it compares the median with the stated value.
Bootstrapped confidence interval for the mean: When the outcome isn't normal but the mean is still the value of interest.

Common mistakes

Choosing the comparison value after seeing the data.
Using a value taken from the same sample, which isn't a fixed comparison.

Introducing the result in a report

Report the sample mean and standard deviation and the comparison value, then the test as t(df) = [value], p = [value], with an effect size such as Cohen's d.

Formulas

Mean
x̄ = Σx ÷ n
Where
x̄
the sample mean
Σx
the sum of all the values
n
the number of values
Add every value and divide by how many there are.
Sample standard deviation
s = √[ Σ(x − x̄)² ÷ (n − 1) ]
Where
s
the sample standard deviation
x
each value
x̄
the sample mean
n
the number of values
The typical distance of values from the mean. Dividing by n − 1 rather than n makes it a better estimate of the population's spread.
One-sample t
t = (x̄ − μ₀) ÷ (s ÷ √n), with n − 1 degrees of freedom
Where
x̄
the sample mean
μ₀
the value the mean is compared with, stated in advance
s
the sample standard deviation
n
the number of participants
How far the sample mean lies from the stated value, measured in standard errors.

Independent-samples t-test

Compares the means of a numeric outcome between two separate groups.

What question it answers

Do two separate groups differ in the mean of a quantitative outcome?

Data it needs

Outcome: One quantitative variable (interval or ratio).
Predictor or grouping variable: One grouping variable with two categories.
Groups: Two.
Independent or paired: Independent: each participant is in one group only.

Grouping variable (independent variable)Bank type

Public banks
Private banks

Outcome (dependent variable)Job satisfaction

Two independent groups compared on one outcomeA grouping variable, bank type, splits participants into two separate groups, public and private; their mean job satisfaction is compared.

Why it may fit

It compares the means of two independent groups on a quantitative outcome.

Why a similar test may not fit

Paired-samples t-test: The paired-samples t-test is for the same participants measured twice, or matched pairs; here each participant is in one group only.
One-way ANOVA: One-way ANOVA is usually used for three or more groups. With two groups it gives the same answer as the equal-variances t-test (F = t²).

Hypotheses

Null hypothesis (H₀): The two population means are equal (μ₁ = μ₂).
Alternative hypothesis (H₁): The two population means differ (μ₁ ≠ μ₂), or, if stated in advance, one is larger.

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Normality: The outcome is approximately normally distributed within each group.
Homogeneity of variance: The groups being compared have similar variances.
No influential outliers: No single case distorts the result.

Example question

Is mean job satisfaction different between public and private bank employees?

Limitations

Only two groups.
Sensitive to outliers in small samples.

Alternatives

Mann–Whitney U test: When the outcome is ordinal, or not normal in small groups.
Welch's t-test: When the groups' variances differ; many recommend it by default.

Common mistakes

Using it when the same participants appear in both groups.
Running a t-test for every pair of three or more groups instead of one ANOVA, which raises the chance of a false positive.
Not saying whether the equal-variances or Welch version was used.

Introducing the result in a report

Report each group's mean and standard deviation, then the test as t(df) = [value], p = [value], with an effect size such as Cohen's d and its confidence interval.

Formulas

Mean
x̄ = Σx ÷ n
Where
x̄
the sample mean
Σx
the sum of all the values
n
the number of values
Add every value and divide by how many there are.
The idea behind every t-test
t = difference ÷ standard error of the difference
Where
difference
the difference being tested, such as between two group means
standard error
how much that difference would vary from sample to sample by chance
A t value compares the difference found with the variation expected by chance. The further t is from zero, the less the difference looks like chance alone.

Paired-samples t-test

Compares the means of a numeric outcome measured twice on the same participants, such as before and after.

What question it answers

Does the mean of a quantitative outcome change between two measurements of the same people?

Data it needs

Outcome: One quantitative variable, measured twice.
Predictor or grouping variable: Time or condition: two measurements.
Groups: One group, two measurements.
Independent or paired: Paired: each participant contributes both measurements, or participants are matched in pairs.

First measurementScore before training

Second measurement, same participantsScore after training

The same participants measured twiceEach participant's score before training is compared with their own score after training.

Why it may fit

It analyses each participant's change, comparing the mean of the differences with zero.

Why a similar test may not fit

Independent-samples t-test: The independent-samples t-test treats the two sets of scores as coming from different people, ignoring that each participant is compared with themselves.

Hypotheses

Null hypothesis (H₀): The mean difference between the paired measurements is zero in the population (μ_d = 0).
Alternative hypothesis (H₁): The mean difference is not zero (μ_d ≠ 0), or, if stated in advance, is positive or negative.

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Paired observations: Each measurement in one condition is matched to one in the other, usually the same participant measured twice.
Normally distributed differences: The differences between each pair of measurements are approximately normal.
No influential outliers: No single case distorts the result.

Example question

Did students' mean scores change after training?

Limitations

Only two measurements.

Alternatives

Wilcoxon signed-rank test: When the differences aren't normal, or the outcome is ordinal.
Bootstrapped confidence interval for the mean difference: When the differences aren't normal but a mean difference is still wanted.

Common mistakes

Checking the normality of each time point instead of the differences.
Using it for two separate groups of people.
Treating a before–after change as the effect of an intervention when there is no comparison group.

Introducing the result in a report

Report the mean and standard deviation at each time, then the test as t(df) = [value], p = [value], with the mean difference and its confidence interval.

Formulas

Paired-samples t
t = d̄ ÷ (s_d ÷ √n), with n − 1 degrees of freedom
Where
d̄
the mean of the differences between each participant's two measurements
s_d
the standard deviation of those differences
n
the number of pairs
A paired t-test is a one-sample t-test on the differences, comparing their mean with zero.

One-way ANOVA

Compares the means of a numeric outcome across three or more independent groups.

What question it answers

Do three or more independent groups differ in the mean of a quantitative outcome?

Data it needs

Outcome: One quantitative variable (interval or ratio).
Predictor or grouping variable: One grouping variable (factor) with three or more categories.
Groups: Three or more.
Independent or paired: Independent: each participant is in one group only.

Grouping variable (factor)Teaching method

Lecture
Discussion
Online

Outcome (dependent variable)Exam score

Three or more independent groups compared on one outcomeA grouping variable, teaching method, forms three separate groups: lecture, discussion and online. Their mean exam scores are compared.

Why it may fit

It compares the variation between group means with the variation within groups, in one test across all the groups.

Why a similar test may not fit

Independent-samples t-test: A t-test for every pair of groups raises the chance of a false positive; ANOVA tests all the groups at once.
Repeated measures ANOVA: Repeated measures ANOVA is for the same participants measured several times, not separate groups.

Hypotheses

Null hypothesis (H₀): All the group means are equal in the population (μ₁ = μ₂ = μ₃ = …).
Alternative hypothesis (H₁): At least one group mean differs from the others.

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Normality: The outcome is approximately normally distributed within each group.
Homogeneity of variance: The groups being compared have similar variances.
No influential outliers: No single case distorts the result.
Adequate group sizes: Each group, or combination of groups, has enough participants.

Example question

Do mean scores differ across three teaching methods?

Limitations

Shows that groups differ, not which ones: post hoc tests are needed.

Alternatives

Kruskal–Wallis test: When the outcome is ordinal, or not normal in small groups.
Welch's ANOVA with Games–Howell post hoc tests: When the groups' variances differ.

Common mistakes

Concluding which groups differ from the F test alone: post hoc tests or planned comparisons are needed.
Using it for repeated measurements of the same people.

Introducing the result in a report

Report each group's mean and standard deviation, then F(df between, df within) = [value], p = [value], with an effect size such as η², followed by post hoc comparisons.

Formulas

The F ratio in ANOVA
F = variance between groups ÷ variance within groups
Where
variance between groups (mean square between)
how far the group means spread around the overall mean
variance within groups (mean square within)
how far participants spread around their own group's mean
When the group means differ by much more than participants differ within groups, F is large.

Two-way ANOVA

Compares means across two grouping factors at once, including whether they interact.

What question it answers

Do two grouping variables, separately and together, relate to differences in a quantitative outcome?

Data it needs

Outcome: One quantitative variable (interval or ratio).
Predictor or grouping variable: Two grouping variables (factors), each with two or more categories.
Groups: Every combination of the two factors' categories.
Independent or paired: Independent: each participant is in one combination only.

Factor ATraining method

Factor BExperience level

Outcome (dependent variable)Sales performance

Interaction: does the effect of training method depend on experience level?

Why it may fit

It tests each factor's main effect and whether the effect of one factor depends on the other: their interaction.

Why a similar test may not fit

One-way ANOVA: Two separate one-way ANOVAs can't test whether the factors interact.

Hypotheses

Null hypothesis (H₀): Three null hypotheses: no difference across the first factor's categories, none across the second's, and no interaction between the factors.
Alternative hypothesis (H₁): For each null hypothesis, the corresponding effect exists in the population.

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Normality: The outcome is approximately normally distributed within each group.
Homogeneity of variance: The groups being compared have similar variances.
No influential outliers: No single case distorts the result.
Adequate group sizes: Each group, or combination of groups, has enough participants.

Example question

Do training method and experience level, separately and together, relate to sales performance?

Limitations

Every combination of groups needs enough participants.

Alternatives

Aligned rank transform ANOVA: When the outcome is ordinal or not normal and the interaction must still be tested.
Robust or bootstrapped ANOVA: When normality or equal variances clearly fail.
Transforming the outcome: When the outcome is strongly skewed.

Common mistakes

Interpreting the main effects without first looking at the interaction.
Leaving some combinations of groups with very few participants.

Introducing the result in a report

Report the mean and standard deviation for each combination, then F and p for each main effect and the interaction, with effect sizes such as partial η²; plot the cell means.

Formulas

The F ratio in ANOVA
F = variance between groups ÷ variance within groups
Where
variance between groups (mean square between)
how far the group means spread around the overall mean
variance within groups (mean square within)
how far participants spread around their own group's mean
When the group means differ by much more than participants differ within groups, F is large.

ANCOVA

Compares group means on a numeric outcome after adjusting for a numeric covariate, such as a control variable or a pretest score.

What question it answers

Do groups differ in the mean of a quantitative outcome once a quantitative covariate is allowed for?

Data it needs

Outcome: One quantitative variable (interval or ratio).
Predictor or grouping variable: One or more grouping variables, plus one or more quantitative covariates.
Groups: Two or more.
Independent or paired: Independent: each participant is in one group only.

Grouping variable (factor)Teaching method

Covariate (quantitative)Prior achievement

Outcome (dependent variable)Final score

Why it may fit

It compares group means adjusted for a covariate that also relates to the outcome, which can account for initial differences and reduce unexplained variation.

Why a similar test may not fit

One-way ANOVA: One-way ANOVA compares unadjusted means, leaving out the covariate.

Hypotheses

Null hypothesis (H₀): The adjusted group means are equal in the population.
Alternative hypothesis (H₁): At least one adjusted group mean differs.

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Normally distributed residuals: The residuals (the differences between observed and predicted values) are approximately normal.
Homogeneity of variance: The groups being compared have similar variances.
Linearity: The relationship between the variables is a straight line.
Homogeneity of regression slopes: The covariate relates to the outcome in the same way in every group.
Covariate independent of the grouping: The covariate isn't affected by group membership, usually because it is measured before the intervention.
No influential outliers: No single case distorts the result.

Example question

Do final scores differ between two teaching methods after controlling for prior achievement?

Limitations

Adjustment can't make non-equivalent groups truly comparable.

Alternatives

Quade's rank analysis of covariance: When the outcome is ordinal or clearly not normal.
Regression with a group × covariate interaction: When the regression slopes differ between groups.

Common mistakes

Skipping the check that the covariate relates to the outcome in the same way in every group.
Using a covariate measured after the groups were formed, which the grouping may have changed.
Expecting adjustment to make groups that differ in other ways truly comparable.

Introducing the result in a report

Report the unadjusted and adjusted means for each group, then F and p for the group effect with the covariate in the model, and an effect size such as partial η².

Formulas

The F ratio in ANOVA
F = variance between groups ÷ variance within groups
Where
variance between groups (mean square between)
how far the group means spread around the overall mean
variance within groups (mean square within)
how far participants spread around their own group's mean
When the group means differ by much more than participants differ within groups, F is large.

Chi-square test of independence

Tests whether two categorical variables are associated.

What question it answers

Are two categorical variables associated?

Data it needs

Outcome: One categorical variable.
Predictor or grouping variable: A second categorical variable.
Groups: Two or more categories of each.
Independent or paired: Independent: each participant is counted once, in one cell.

First variable (categorical)Gender

Second variable (categorical)Employment status

Two categorical variables examined togetherGender and employment status, both categorical, are cross-tabulated to see whether they are associated.

Why it may fit

It compares the counts in a cross-tabulation with the counts expected if the two variables were unrelated.

Why a similar test may not fit

Independent-samples t-test: A t-test compares means, which need a quantitative outcome; here both variables are categories.

Hypotheses

Null hypothesis (H₀): The two variables are independent in the population.
Alternative hypothesis (H₁): The two variables are associated.

Key assumptions

Categorical variables: Both variables are measured in categories.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Adequate expected counts: The counts expected in each cell, if the variables were unrelated, are large enough.

Example question

Is employment status associated with gender?

Limitations

Shows association, not its strength or cause; report an effect size too.

Alternatives

Fisher's exact test: When expected counts are too small for chi-square.
Combining sparse categories: When some categories have very few cases and theory allows merging them.

Common mistakes

Calculating it from percentages instead of counts.
Counting the same participant in more than one cell.
Reading a clear result as cause, or as a strong association: report an effect size such as Cramér's V.

Introducing the result in a report

Report the cross-tabulated counts and percentages, then χ²(df, N = [total]) = [value], p = [value], with Cramér's V.

Formulas

Expected count in a cross-tabulation
E = (row total × column total) ÷ N
Where
E
the count expected in a cell if the two variables were unrelated
N
the total number of participants
If the variables were unrelated, each cell would hold its row's share of its column's total.
Chi-square statistic
χ² = Σ (O − E)² ÷ E
Where
χ²
the chi-square statistic
O
the observed count in a cell or category
E
the expected count in the same cell or category
Σ
add the result for every cell or category
The larger the gaps between observed and expected counts, relative to the expected counts, the larger χ².

Chi-square goodness-of-fit test

Tests whether the counts in one categorical variable's categories match expected proportions, such as equal shares or known population shares.

What question it answers

Do the counts in one categorical variable's categories match shares stated in advance?

Data it needs

Outcome: One categorical variable.
Predictor or grouping variable: None: expected proportions stated in advance.
Groups: Two or more categories.
Independent or paired: Independent: each participant falls in one category.

Categorical variable (observed counts)Service channel chosen

Expected counts (stated in advance)Equal shares for every channel

One categorical variable compared with expected sharesThe counts of customers choosing each service channel are compared with the counts expected if every channel were chosen equally often.

Why it may fit

It compares the observed count in each category with the count expected from shares stated in advance.

Why a similar test may not fit

Chi-square test of independence: The chi-square test of independence needs two categorical variables; here there is one, compared with expected shares.

Hypotheses

Null hypothesis (H₀): The population proportions equal the stated expected proportions.
Alternative hypothesis (H₁): At least one population proportion differs from its expected proportion.

Key assumptions

Independent observations.
Each participant falls in exactly one category.
Expected counts of at least 5 in most categories.

The Statistical Assumption Checker doesn't have a guide for this test yet; these assumptions come from the analysis catalogue.

Example question

Do customers choose the bank's four service channels equally often?

Limitations

The expected proportions must be stated before looking at the data.
Shows that counts differ from expectation, not why.

Alternatives

Common mistakes

Setting the expected proportions after looking at the observed counts.
Calculating it from percentages instead of counts.

Introducing the result in a report

Report the observed and expected count for each category, then χ²(df, N = [total]) = [value], p = [value].

Formulas

Chi-square statistic
χ² = Σ (O − E)² ÷ E
Where
χ²
the chi-square statistic
O
the observed count in a cell or category
E
the expected count in the same cell or category
Σ
add the result for every cell or category
The larger the gaps between observed and expected counts, relative to the expected counts, the larger χ².

Pearson correlation

Measures the strength and direction of a straight-line relationship between two numeric variables.

What question it answers

Do two quantitative variables go together in a straight line, and how strongly?

Data it needs

Outcome: First variable: quantitative.
Predictor or grouping variable: Second variable: quantitative. Neither needs to be the outcome.
Groups: Not applicable: one group, a pair of values per participant.
Independent or paired: Each participant contributes one value of each variable.

First variable (quantitative)Study time

Second variable (quantitative)Exam score

Two quantitative variables examined togetherStudy time and exam score are examined together, with no variable treated as the outcome.

Why it may fit

It measures the strength and direction of a straight-line relationship between two quantitative variables.

Why a similar test may not fit

Simple linear regression: Regression goes further, estimating one variable from the other; correlation treats the two alike.
Spearman's rank correlation: Spearman's rank correlation suits ordinal variables, curved but consistently rising or falling relationships, or clearly non-normal data.

Hypotheses

Null hypothesis (H₀): There is no straight-line relationship in the population (ρ = 0).
Alternative hypothesis (H₁): There is a straight-line relationship (ρ ≠ 0), or, if stated in advance, a positive or negative one.

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Linearity: The relationship between the variables is a straight line.
Normality: The outcome is approximately normally distributed within each group.
No influential outliers: No single case distorts the result.

Example question

Is study time associated with examination score?

Limitations

It misses curved relationships.
Outliers can distort it strongly.

Alternatives

Spearman's rank correlation: When a variable is ordinal, the relationship is monotonic but curved, or the data aren't normal.
Kendall's tau-b: When there are many tied ranks or the sample is small.

Common mistakes

Treating a correlation as evidence of cause and effect.
Not looking at the scatterplot, which shows curves and outliers the coefficient hides.
Studying a narrow range of either variable, which weakens the correlation.

Introducing the result in a report

Report r(df) = [value], p = [value], with the confidence interval for r, and show the scatterplot.

Simple linear regression

Predicts a numeric outcome from one variable.

What question it answers

How well can a quantitative outcome be predicted from one predictor, and how much does it change per unit of the predictor?

Data it needs

Outcome: One quantitative variable.
Predictor or grouping variable: One predictor, usually quantitative.
Groups: Not applicable.
Independent or paired: Each participant contributes one value of each variable.

Predictor (independent variable)Study hours

Outcome (dependent variable)Exam score

One predictor used to estimate an outcomeStudy hours, the predictor, points to exam score, the outcome being predicted.

Why it may fit

It estimates a straight-line equation that predicts the outcome from the predictor.

Why a similar test may not fit

Pearson correlation: Correlation describes how strongly the variables go together but gives no prediction equation.

Hypotheses

Null hypothesis (H₀): The slope is zero in the population (β₁ = 0): the predictor doesn't predict the outcome in a straight line.
Alternative hypothesis (H₁): The slope is not zero (β₁ ≠ 0).

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Linearity: The relationship between the variables is a straight line.
Normally distributed residuals: The residuals (the differences between observed and predicted values) are approximately normal.
Homoscedasticity: The spread of the residuals is the same at every level of the predicted values.
Independent errors: The residuals of any two observations are unrelated, such as over time or in the order data were collected.
No influential outliers: No single case distorts the result.

Example question

Can examination score be predicted from study hours?

Limitations

Leaves out other variables that may matter.

Alternatives

Spearman's rank correlation: When only the strength of a monotonic relationship is needed, not a prediction equation.
Robust (heteroscedasticity-consistent) standard errors: When the residuals' spread is unequal.
Bootstrapped confidence intervals: When the residuals aren't normal, especially in smaller samples.
Transforming the outcome: When the outcome is strongly skewed or the relationship is curved.

Common mistakes

Reading the slope as a causal effect in an observational study.
Predicting far beyond the range of the data.
Checking the normality of each variable instead of the residuals.

Introducing the result in a report

Report the slope with its confidence interval and p, the intercept, and R², for example b = [value], 95% CI [lower, upper], R² = [value].

Formulas

Simple linear regression
Y = β₀ + β₁X + ε
Where
Y
the outcome
X
the predictor
β₀
the intercept, the predicted outcome when X is zero
β₁
the slope, the change in the predicted outcome for each one-unit increase in X
ε
the error, what the line doesn't explain for each participant
The outcome is modelled as a straight line in the predictor, plus what the line leaves unexplained.

Multiple regression

Predicts a numeric outcome from several variables at once, each adjusted for the others.

What question it answers

How well do several predictors together predict a quantitative outcome, and what does each add while the others are held constant?

Data it needs

Outcome: One quantitative variable.
Predictor or grouping variable: Two or more predictors, quantitative or categorical (entered as dummy variables).
Groups: Not applicable.
Independent or paired: Each participant contributes one value of each variable.

Predictor 1Study hours

Predictor 2Attendance

Outcome (dependent variable)Exam score

Several predictors used together to estimate an outcomeTwo predictors, study hours and attendance, both point to exam score. Each one's contribution is estimated while holding the other constant.

Why it may fit

It estimates each predictor's contribution to the outcome while holding the other predictors constant.

Why a similar test may not fit

Simple linear regression: Simple regression uses one predictor and can't separate the contributions of related predictors.

Hypotheses

Null hypothesis (H₀): Overall, none of the predictors predicts the outcome (all slopes are zero); and, for each predictor, its slope is zero.
Alternative hypothesis (H₁): At least one slope is not zero; and, for each predictor, its slope is not zero.

Key assumptions

Interval or ratio measurement: The outcome is measured on a numeric scale with equal intervals between values.
Independence of observations: Each participant's data are unrelated to every other participant's, and each participant contributes once.
Linearity: The relationship between the variables is a straight line.
Normally distributed residuals: The residuals (the differences between observed and predicted values) are approximately normal.
Homoscedasticity: The spread of the residuals is the same at every level of the predicted values.
Independent errors: The residuals of any two observations are unrelated, such as over time or in the order data were collected.
No influential outliers: No single case distorts the result.
No multicollinearity: The predictors aren't too highly correlated with each other.

Example question

Can examination score be predicted from study hours?

Limitations

Needs more participants as predictors are added.
Highly related predictors make individual estimates unstable.

Alternatives

Robust (heteroscedasticity-consistent) standard errors: When the residuals' spread is unequal.
Bootstrapped confidence intervals: When the residuals aren't normal, especially in smaller samples.
Transforming the outcome: When the outcome is strongly skewed or the relationship is curved.

Common mistakes

Adding many predictors to a small sample.
Interpreting coefficients without checking that the predictors aren't too highly related (multicollinearity).
Reading coefficients as causal effects in an observational study.

Introducing the result in a report

Report R² and the overall F test, then each predictor's coefficient with its confidence interval and p, usually in a table.

Formulas

Multiple linear regression
Y = β₀ + β₁X₁ + β₂X₂ + … + βₖXₖ + ε
Where
Y
the outcome
X₁ … Xₖ
the k predictors
β₀
the intercept
β₁ … βₖ
each predictor's slope, holding the other predictors constant
ε
the error
Each slope estimates a predictor's contribution while the other predictors are held constant.

Follow worked examples from question to method

Each example uses a small, fixed illustrative dataset. The tool calculates the displayed summaries from those values so a reader can reproduce the arithmetic. These examples demonstrate how to describe the design and identify a candidate; their small samples are not used to claim statistical significance or to teach a complete inferential calculation.

Research questionIs the mean weekly working time of bank tellers different from the 40 hours set in their contracts?

Management research

Variables

Weekly working hours: Outcome, ratio level. Hours worked in a week, with a true zero.

Groups

One group of tellers, compared with a fixed value of 40 hours.

Structure

There is no second group: the comparison value comes from the contract, stated before the data were collected.

Outcome (quantitative)Weekly working hours

Comparison value (stated in advance)40 hours

One sample compared with a stated valueThe mean of one quantitative variable, weekly working hours, is compared with a value stated in advance: the 40 hours in the contract.

A candidate test

One-sample t-test. One quantitative outcome is compared with a value stated in advance, which is what the one-sample t-test does.

Assumptions to check

Interval or ratio measurement
Independence of observations
Normality
No influential outliers

Weekly working hours of six tellers (invented for illustration) Scroll sideways to see every column.
Teller	Hours
1	41
2	45
3	40
4	44
5	46
6	42

Worked from the data

Mean = (41 + 45 + 40 + 44 + 46 + 42) ÷ 6 = 258 ÷ 6 = 43.0 hours, against the contract's 40.
Standard deviation s = 2.37, so t = (43.0 − 40) ÷ (2.37 ÷ √6) = 3.11 with 5 degrees of freedom.

What the result would tell you

The test asks whether the sample mean is further from 40 than chance variation between samples would usually produce. A difference can matter to managers even when it is small, or be statistically clear yet too small to matter.

Research questionIs mean job satisfaction different between public and private bank employees?

Business research

Variables

Bank type: Grouping variable (independent variable), nominal level. Two categories: public and private.
Job satisfaction: Outcome (dependent variable), interval level. A scale score from ten rating items, commonly treated as interval.

Groups

Two groups: public and private bank employees.

Structure

Independent: each employee works in one type of bank only.

Grouping variable (independent variable)Bank type

Public banks
Private banks

Outcome (dependent variable)Job satisfaction

Two independent groups compared on one outcomeA grouping variable, bank type, splits participants into two separate groups, public and private; their mean job satisfaction is compared.

A candidate test

Independent-samples t-test. A quantitative outcome is compared between two independent groups, which is the situation the independent-samples t-test is built for.

Assumptions to check

Interval or ratio measurement
Independence of observations
Normality
Homogeneity of variance
No influential outliers

Job satisfaction scores out of 50 (invented for illustration) Scroll sideways to see every column.
Employee	Bank type	Satisfaction
1	Public	38
2	Public	34
3	Public	40
4	Public	36
5	Private	32
6	Private	35
7	Private	30
8	Private	31

Worked from the data

Public: mean = (38 + 34 + 40 + 36) ÷ 4 = 148 ÷ 4 = 37.0.
Private: mean = (32 + 35 + 30 + 31) ÷ 4 = 128 ÷ 4 = 32.0.
The sample means differ by 5.0 points. With four employees per group this is an illustration only; the test weighs the gap against the spread within each group.

What the result would tell you

The test asks whether the gap between the group means is larger than chance would usually produce. Report the effect size as well: a clear difference can still be small in practice.

Research questionDid students' mean scores change after training?

Education research

Variables

Score before training: First measurement, interval level. A test score out of 100.
Score after training: Second measurement, same students, interval level. The same test, taken again.

Groups

One group of students, measured twice.

Structure

Paired: every “after” score belongs to the same student as one “before” score.

First measurementScore before training

Second measurement, same participantsScore after training

The same participants measured twiceEach participant's score before training is compared with their own score after training.

A candidate test

Paired-samples t-test. The same students are measured twice on a quantitative outcome, so the paired-samples t-test analyses each student's change.

Assumptions to check

Interval or ratio measurement
Paired observations
Normally distributed differences
No influential outliers

Test scores before and after training (invented for illustration) Scroll sideways to see every column.
Student	Before	After	Difference
1	62	68	6
2	55	60	5
3	70	74	4
4	58	59	1
5	65	71	6

Worked from the data

Mean before = 62.0; mean after = 66.4.
Mean difference d̄ = (6 + 5 + 4 + 1 + 6) ÷ 5 = 4.4, with s_d = 2.07, so t = 4.4 ÷ (2.07 ÷ √5) = 4.74 with 4 degrees of freedom.

What the result would tell you

The test asks whether the mean change is further from zero than chance would usually produce. Without a comparison group it can't show that the training caused the change: practice, time or other events may explain it.

Research questionIs employment status associated with gender?

Social science research

Variables

Gender: First categorical variable, nominal level. Female or male, as recorded in this survey.
Employment status: Second categorical variable, nominal level. Employed or unemployed.

Groups

Two categories of each variable, making a 2 × 2 table.

Structure

Independent: each respondent is counted once, in one cell.

First variable (categorical)Gender

Second variable (categorical)Employment status

Two categorical variables examined togetherGender and employment status, both categorical, are cross-tabulated to see whether they are associated.

A candidate test

Chi-square test of independence. Both variables are categories, so the question is whether the counts in the table depart from what unrelated variables would give.

Assumptions to check

Categorical variables
Independence of observations
Adequate expected counts

Observed counts, with expected counts in brackets (invented for illustration) Scroll sideways to see every column.
Gender	Employed	Unemployed	Total
Female	42 (45)	18 (15)	60
Male	48 (45)	12 (15)	60
Total	90	30	120

Worked from the data

Expected count for employed women = (60 × 90) ÷ 120 = 45; the other cells follow the same way.
χ² = (42 − 45)² ÷ 45 + (18 − 15)² ÷ 15 + (48 − 45)² ÷ 45 + (12 − 15)² ÷ 15 = 1.60, with 1 degree of freedom.

What the result would tell you

The test asks whether the pattern of counts differs from independence by more than chance would usually produce. It says nothing about cause, and a clear association can still be weak; Cramér's V describes its strength.

Research questionDo customers choose the bank's four service channels equally often?

Business research

Variables

Service channel chosen: Categorical variable, nominal level. Branch, mobile app, ATM or internet banking.

Groups

Four categories, compared with equal expected shares.

Structure

Independent: each customer's most recent transaction is counted once.

Categorical variable (observed counts)Service channel chosen

Expected counts (stated in advance)Equal shares for every channel

One categorical variable compared with expected sharesThe counts of customers choosing each service channel are compared with the counts expected if every channel were chosen equally often.

A candidate test

Chi-square goodness-of-fit test. One categorical variable is compared with shares stated in advance, which is what the goodness-of-fit test does.

Assumptions to check

Independent observations.
Each participant falls in exactly one category.
Expected counts of at least 5 in most categories.

Channel used by 120 customers (invented for illustration) Scroll sideways to see every column.
Channel	Observed	Expected
Branch	38	30
Mobile app	42	30
ATM	22	30
Internet banking	18	30

Worked from the data

Equal shares give 120 ÷ 4 = 30 expected customers per channel.
χ² = Σ (O − E)² ÷ E = 13.87, with 3 degrees of freedom (four categories minus one).

What the result would tell you

The test asks whether the counts depart from equal shares by more than chance would usually produce. It doesn't say which channels differ, or why; look at the gap between observed and expected counts in each category.

Research questionIs study time associated with examination score?

Education research

Variables

Study time: First variable, ratio level. Hours of study per week.
Examination score: Second variable, interval level. A score out of 100.

Groups

One group of students, each with a value on both variables.

Structure

Each student contributes one pair of values; no variable is treated as the outcome.

First variable (quantitative)Study time

Second variable (quantitative)Exam score

Two quantitative variables examined togetherStudy time and exam score are examined together, with no variable treated as the outcome.

A candidate test

Pearson correlation. Two quantitative variables are examined together for a straight-line relationship, which Pearson's correlation measures.

Assumptions to check

Interval or ratio measurement
Independence of observations
Linearity
Normality
No influential outliers

Study time and exam score for six students (invented for illustration) Scroll sideways to see every column.
Student	Study hours	Exam score
1	2	55
2	4	56
3	5	63
4	7	64
5	8	72
6	10	74

Worked from the data

Pearson's r = 0.96 for these six illustrative students: a strong positive straight-line relationship in this sample.

What the result would tell you

A positive correlation means that students who study more tend to score higher. It doesn't show that studying causes higher scores: ability or motivation could affect both.

Research questionCan examination score be predicted from study hours?

Education research

Variables

Study hours: Predictor (independent variable), ratio level. Hours of study per week.
Examination score: Outcome (dependent variable), interval level. A score out of 100.

Groups

One group of students.

Structure

Each student contributes one value of each variable; study hours predict the score.

Predictor (independent variable)Study hours

Outcome (dependent variable)Exam score

One predictor used to estimate an outcomeStudy hours, the predictor, points to exam score, the outcome being predicted.

A candidate test

Simple linear regression. One quantitative predictor is used to estimate a quantitative outcome with a straight-line equation, which is simple linear regression.

Assumptions to check

Interval or ratio measurement
Independence of observations
Linearity
Normally distributed residuals
Homoscedasticity
Independent errors
No influential outliers

The same six students as the correlation example (invented for illustration) Scroll sideways to see every column.
Student	Study hours	Exam score
1	2	55
2	4	56
3	5	63
4	7	64
5	8	72
6	10	74

Worked from the data

Least-squares line: predicted score = 48.4 + 2.60 × study hours.
So each extra hour of study goes with a predicted score about 2.6 points higher in these data. Correlation and regression use the same data but answer different questions: how strongly they go together, and what score to predict.

What the result would tell you

The slope says how much the predicted score changes for each extra hour of study. It describes prediction in this sample, not the effect of making a student study more.

Research questionDo mean scores differ across three teaching methods?

Education research

Variables

Teaching method: Grouping variable (factor), nominal level. Lecture, discussion or online.
Exam score: Outcome (dependent variable), interval level. A score out of 100.

Groups

Three groups: lecture, discussion and online.

Structure

Independent: each student is taught by one method only.

Grouping variable (factor)Teaching method

Lecture
Discussion
Online

Outcome (dependent variable)Exam score

Three or more independent groups compared on one outcomeA grouping variable, teaching method, forms three separate groups: lecture, discussion and online. Their mean exam scores are compared.

A candidate test

One-way ANOVA. A quantitative outcome is compared across three independent groups in one test, which is what one-way ANOVA does.

Assumptions to check

Interval or ratio measurement
Independence of observations
Normality
Homogeneity of variance
No influential outliers
Adequate group sizes

Exam scores under three teaching methods (invented for illustration) Scroll sideways to see every column.
Student	Method	Score
1	Lecture	65
2	Lecture	70
3	Lecture	60
4	Discussion	75
5	Discussion	72
6	Discussion	78
7	Online	68
8	Online	64
9	Online	66

Worked from the data

Lecture: mean = (65 + 70 + 60) ÷ 3 = 195 ÷ 3 = 65.0.
Discussion: mean = (75 + 72 + 78) ÷ 3 = 225 ÷ 3 = 75.0.
Online: mean = (68 + 64 + 66) ÷ 3 = 198 ÷ 3 = 66.0.
Mean square between = 91.0; mean square within = 12.67; F = 91.0 ÷ 12.67 = 7.18, with 2 and 6 degrees of freedom.

What the result would tell you

A large F says the group means differ more than the variation within groups would lead you to expect. It doesn't say which groups differ: post hoc tests or planned comparisons answer that.

Research questionDo training method and experience level, separately and together, relate to sales performance?

Management research

Variables

Training method: Factor A, nominal level. Online or classroom.
Experience level: Factor B, nominal level. Junior or senior.
Sales performance: Outcome (dependent variable), ratio level. Units sold in a month.

Groups

Four combinations: online junior, online senior, classroom junior and classroom senior.

Structure

Independent: each salesperson is in one combination only.

Factor ATraining method

Factor BExperience level

Outcome (dependent variable)Sales performance

Interaction: does the effect of training method depend on experience level?

A candidate test

Two-way ANOVA. Two grouping variables and one quantitative outcome are analysed together, including whether they interact, which is what two-way ANOVA does.

Assumptions to check

Interval or ratio measurement
Independence of observations
Normality
Homogeneity of variance
No influential outliers
Adequate group sizes

Mean units sold in each combination (two salespeople per cell, invented for illustration) Scroll sideways to see every column.
Training method	Junior	Senior
Online	21	31
Classroom	29	33

Worked from the data

For juniors, classroom training is 8 units ahead of online training; for seniors, only 2 units ahead.
Because the gap between methods changes with experience level, these illustrative means show an interaction pattern. Two-way ANOVA tests whether a pattern like this is larger than chance would usually produce.

What the result would tell you

Two-way ANOVA tests three things: a difference between training methods, a difference between experience levels, and an interaction. An interaction means the effect of one factor depends on the other; when there is one, describe the effects within each level rather than relying on the overall main effects.

Research questionDo final scores differ between two teaching methods after controlling for prior achievement?

Education research

Variables

Teaching method: Grouping variable (factor), nominal level. Method A or method B.
Prior achievement: Covariate, interval level. A pretest score, measured before teaching began.
Final score: Outcome (dependent variable), interval level. A score out of 100.

Groups

Two groups, one per teaching method.

Structure

Independent: each student is taught by one method; the covariate was measured before the groups were taught.

Grouping variable (factor)Teaching method

Covariate (quantitative)Prior achievement

Outcome (dependent variable)Final score

A candidate test

ANCOVA. Groups are compared on a quantitative outcome while adjusting for a quantitative covariate, which is what ANCOVA does.

Assumptions to check

Interval or ratio measurement
Independence of observations
Normally distributed residuals
Homogeneity of variance
Linearity
Homogeneity of regression slopes
Covariate independent of the grouping
No influential outliers

Prior achievement and final scores (invented for illustration) Scroll sideways to see every column.
Student	Method	Prior achievement	Final score
1	Method A	60	65
2	Method A	70	74
3	Method A	80	83
4	Method B	50	60
5	Method B	60	69
6	Method B	70	78

Worked from the data

Method A: mean prior achievement = 70.0; mean final score = 74.0.
Method B: mean prior achievement = 60.0; mean final score = 69.0.
Method B's final mean is 5.0 points lower, but its students also started 10.0 points lower. ANCOVA asks whether the methods still differ once that starting gap is allowed for.

What the result would tell you

ANCOVA compares the groups' final scores as if they had started with the same prior achievement. Adjustment helps, but it can't make groups that differ in other ways truly comparable; random assignment does that.

What the formulas compare, and what results mean

A test statistic expresses how the observed pattern compares with variation expected under a null hypothesis. The formulas below show the ideas behind means, t statistics, expected counts, chi-square, regression and the F ratio; every symbol is defined. Software can calculate these quantities, but the researcher still has to select a defensible method and explain the result. Standardized mean differences and effect-size reporting conventions are described by Hedges (1981) and Lakens (2013).

Mean
x̄ = Σx ÷ n
Where
x̄
the sample mean
Σx
the sum of all the values
n
the number of values
Add every value and divide by how many there are.
Sample standard deviation
s = √[ Σ(x − x̄)² ÷ (n − 1) ]
Where
s
the sample standard deviation
x
each value
x̄
the sample mean
n
the number of values
The typical distance of values from the mean. Dividing by n − 1 rather than n makes it a better estimate of the population's spread.
The idea behind every t-test
t = difference ÷ standard error of the difference
Where
difference
the difference being tested, such as between two group means
standard error
how much that difference would vary from sample to sample by chance
A t value compares the difference found with the variation expected by chance. The further t is from zero, the less the difference looks like chance alone.
One-sample t
t = (x̄ − μ₀) ÷ (s ÷ √n), with n − 1 degrees of freedom
Where
x̄
the sample mean
μ₀
the value the mean is compared with, stated in advance
s
the sample standard deviation
n
the number of participants
How far the sample mean lies from the stated value, measured in standard errors.
Paired-samples t
t = d̄ ÷ (s_d ÷ √n), with n − 1 degrees of freedom
Where
d̄
the mean of the differences between each participant's two measurements
s_d
the standard deviation of those differences
n
the number of pairs
A paired t-test is a one-sample t-test on the differences, comparing their mean with zero.
Expected count in a cross-tabulation
E = (row total × column total) ÷ N
Where
E
the count expected in a cell if the two variables were unrelated
N
the total number of participants
If the variables were unrelated, each cell would hold its row's share of its column's total.
Chi-square statistic
χ² = Σ (O − E)² ÷ E
Where
χ²
the chi-square statistic
O
the observed count in a cell or category
E
the expected count in the same cell or category
Σ
add the result for every cell or category
The larger the gaps between observed and expected counts, relative to the expected counts, the larger χ².
Simple linear regression
Y = β₀ + β₁X + ε
Where
Y
the outcome
X
the predictor
β₀
the intercept, the predicted outcome when X is zero
β₁
the slope, the change in the predicted outcome for each one-unit increase in X
ε
the error, what the line doesn't explain for each participant
The outcome is modelled as a straight line in the predictor, plus what the line leaves unexplained.
Multiple linear regression
Y = β₀ + β₁X₁ + β₂X₂ + … + βₖXₖ + ε
Where
Y
the outcome
X₁ … Xₖ
the k predictors
β₀
the intercept
β₁ … βₖ
each predictor's slope, holding the other predictors constant
ε
the error
Each slope estimates a predictor's contribution while the other predictors are held constant.
The F ratio in ANOVA
F = variance between groups ÷ variance within groups
Where
variance between groups (mean square between)
how far the group means spread around the overall mean
variance within groups (mean square within)
how far participants spread around their own group's mean
When the group means differ by much more than participants differ within groups, F is large.

A p-value is not the probability that the null hypothesis is true, nor a measure of the size or practical importance of an effect. Interpret it with the study design, uncertainty intervals, an appropriate effect size, the measurement context and the research question. Effect-size benchmarks are context-dependent guides, not universal cut-offs (Cohen, 1988; Wasserstein & Lazar, 2016).

Common mistakes and limits of a decision guide

Choosing a test from the research topic alone, without identifying the outcome, predictors and study design.
Treating ordinal categories as if equal distances between their values were guaranteed.
Using separate-group methods for repeated measurements, or paired methods without genuinely matched observations.
Running many pairwise t-tests after an ANOVA without addressing the increased chance of false-positive findings.
Treating association or prediction as proof of cause; causal claims require a design and assumptions that support them.
Checking normality only with a significance test, or checking the wrong quantity: paired differences for paired t-tests, and residuals for regression.
Reporting only a p-value while omitting descriptive statistics, effect size, uncertainty and the analysis choices that matter.
Treating a suggested test as uniquely correct. Small samples, complex designs, missing data, clustering, survey weights, non-linear patterns and unusual distributions may require methods outside this guide.

This guide covers common introductory methods, not every valid analysis. It does not inspect a dataset, determine whether an assumption holds, calculate a sample-size-specific power analysis, or replace a supervisor's methodological judgement. When several candidates remain plausible, explain the choice and its assumptions rather than forcing certainty.

References

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Field, A. (2018). Discovering statistics using IBM SPSS statistics (5th ed.). SAGE.
Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.3102/10769986006002107 (external link)
Howell, D. C. (2013). Statistical methods for psychology (8th ed.). Wadsworth, Cengage Learning.
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, Article 863. https://doi.org/10.3389/fpsyg.2013.00863 (external link)
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108 (external link)

Frequently asked questions

Can the finder identify the one correct test?

Not in every study. It narrows the options using your stated purpose, variable types and design, then explains what may fit. Complex sampling, missing data, small samples or unmet assumptions may change the analysis, so discuss uncertain choices with a supervisor.

Does a significant result show that an effect matters?

No. Statistical evidence and practical importance are different. Report an appropriate effect size and uncertainty, and interpret them in the context of the research question and study design.

Should I test normality before collecting data?

Usually the distribution cannot be known before data exist. Plan the analysis from the design, then inspect the relevant distribution or residuals when data are available. For a paired t-test, assess the differences.

What should I do if two candidate tests may fit?

Compare the assumptions, estimand and limitations of each, and state why your chosen method answers the research question. A supervisor or statistician can help when the design is complex.

Ready to apply this?

Put what you have read into practice with Statistical Test Finder.

Open Statistical Test Finder