Statistical Assumption Checker
Before running an analysis, check that its assumptions hold. The checker lists the assumptions behind each analysis your project plans, says what your project already shows about them, and explains how to check the rest once you have data. It never tests assumptions itself.
Your project
Enter what you already know. The checklist follows from the analyses your project plans.
One per line.
One variable per line. Put its indicators after a colon, separated by semicolons, such as “wellbeing: life satisfaction; positive mood”.
A plan from the Sample Size Calculator, using Cochran's formula at 95% confidence.
Your assumption checklist
The analyses your project plans, and the assumptions to check before running each.
Descriptive statistics Strong recommendation
Used for: Describe the participants and each variable.
Summaries that match the distribution For you to review
It can only be checked once the data are collected. Inspect a histogram or bar chart of each variable.
No influential outliers For you to review
It can only be checked once the data are collected. Inspect boxplots and scatterplots.
Notes
- Record your variables and how they are measured to see which analyses, and so which assumptions, apply.
- Check assumptions before running each analysis, and report the checks and any corrections in your method or results section.
Assumptions by analysis
Every analysis the checker covers, and everything about its assumptions. Thresholds are commonly used conventions; references will be added after academic review.
Descriptive statistics
Summarises who took part and what each variable looks like before any test.
Assumptions to check
Summaries that match the distribution
Each summary suits the variable's measurement level and shape.
Why it matters
A mean describes a symmetric numeric variable well but misleads for skewed data or single ordinal items.
How to check it
- Inspect a histogram or bar chart of each variable.
- Compare the mean with the median.
Commonly used thresholds
Report the mean and standard deviation for roughly symmetric numeric variables, and the median and interquartile range for skewed or ordinal ones.
If it is violated
Readers get a misleading picture of what is typical.
What to do instead
- Report the median and interquartile range.
- Report frequencies and percentages for categories.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Non-parametric alternatives
- Median: When a numeric variable is skewed, or a variable is ordinal: report the median and interquartile range.
Reporting example
[Variable] was positively skewed (skewness = 1.84), so the median and interquartile range are reported (Mdn = 12, IQR = 8–19).
Illustrative numbers; replace them with your own.
Common mistakes
- Reporting means for single rating items or skewed variables without comment.
- Not checking the data for entry errors, such as impossible values, before summarising.
References awaiting academic review.
Correlation
Measures how strongly two variables move together.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Linearity
The relationship between the variables is a straight line.
Why it matters
Pearson correlation and linear regression measure straight-line relationships; a curved one is underestimated or missed.
How to check it
- Inspect a scatterplot of each pair.
- In regression, plot residuals against predicted values: they should scatter evenly around zero with no curve.
Commonly used thresholds
- No numeric threshold: judge the plots.
- A clear curve in the residual plot means the relationship isn't linear.
If it is violated
The strength of the relationship is underestimated, and predictions are wrong in parts of the range.
What to do instead
- Add a squared term, or transform a variable.
- Use Spearman's correlation for a relationship that is monotonic but curved.
Normality
The outcome is approximately normally distributed within each group.
Why it matters
Parametric tests use the normal distribution to judge how surprising a result is; strong non-normality, especially in small samples, makes p-values inaccurate.
How to check it
- Inspect histograms and Q–Q plots for each group.
- Check skewness and kurtosis.
- Run the Shapiro–Wilk test, bearing in mind its power depends on sample size.
Commonly used thresholds
- Shapiro–Wilk p above .05 is consistent with normality; with large samples it flags trivial departures, so rely on plots.
- Skewness and kurtosis between −2 and +2 are often treated as acceptable; some researchers use −1 to +1.
If it is violated
With small samples, p-values and confidence intervals can be wrong. With larger samples, tests of means are commonly considered robust to moderate non-normality.
What to do instead
- Use the non-parametric alternative.
- Transform the variable, such as a log transformation for right skew, and report it.
- Use bootstrapped confidence intervals.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Kendall's tau-b: When there are many tied ranks or the sample is small.
Non-parametric alternatives
- Spearman's rank correlation: When a variable is ordinal, the relationship is monotonic but curved, or the data aren't normal.
Reporting example
Scatterplots showed a linear relationship with no extreme outliers, and both variables were approximately normal (skewness and kurtosis within ±1), so Pearson's correlation was used.
Illustrative numbers; replace them with your own.
Common mistakes
- Calculating a correlation without looking at the scatterplot.
- Choosing between Pearson and Spearman after seeing which gives the significant result.
References awaiting academic review.
Pearson correlation
Measures the strength and direction of a straight-line relationship between two numeric variables.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Linearity
The relationship between the variables is a straight line.
Why it matters
Pearson correlation and linear regression measure straight-line relationships; a curved one is underestimated or missed.
How to check it
- Inspect a scatterplot of each pair.
- In regression, plot residuals against predicted values: they should scatter evenly around zero with no curve.
Commonly used thresholds
- No numeric threshold: judge the plots.
- A clear curve in the residual plot means the relationship isn't linear.
If it is violated
The strength of the relationship is underestimated, and predictions are wrong in parts of the range.
What to do instead
- Add a squared term, or transform a variable.
- Use Spearman's correlation for a relationship that is monotonic but curved.
Normality
The outcome is approximately normally distributed within each group.
Why it matters
Parametric tests use the normal distribution to judge how surprising a result is; strong non-normality, especially in small samples, makes p-values inaccurate.
How to check it
- Inspect histograms and Q–Q plots for each group.
- Check skewness and kurtosis.
- Run the Shapiro–Wilk test, bearing in mind its power depends on sample size.
Commonly used thresholds
- Shapiro–Wilk p above .05 is consistent with normality; with large samples it flags trivial departures, so rely on plots.
- Skewness and kurtosis between −2 and +2 are often treated as acceptable; some researchers use −1 to +1.
If it is violated
With small samples, p-values and confidence intervals can be wrong. With larger samples, tests of means are commonly considered robust to moderate non-normality.
What to do instead
- Use the non-parametric alternative.
- Transform the variable, such as a log transformation for right skew, and report it.
- Use bootstrapped confidence intervals.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Kendall's tau-b: When there are many tied ranks or the sample is small.
Non-parametric alternatives
- Spearman's rank correlation: When a variable is ordinal, the relationship is monotonic but curved, or the data aren't normal.
Reporting example
Scatterplots showed a linear relationship with no extreme outliers, and both variables were approximately normal (Shapiro–Wilk p = .21 and p = .34).
Illustrative numbers; replace them with your own.
Common mistakes
- Calculating Pearson's r without looking at the scatterplot.
- Ignoring a single outlier that creates, or hides, the correlation.
References awaiting academic review.
Spearman's rank correlation
Measures how consistently two variables rise or fall together, using ranks.
Assumptions to check
At least ordinal measurement
Values can be put in a meaningful order.
Why it matters
Rank-based methods compare orders; unordered categories have no order to compare.
How to check it
Check each variable's measurement level.
Commonly used thresholds
No numeric threshold: ordinal, interval and ratio variables all qualify.
If it is violated
Ranks of unordered categories are meaningless.
What to do instead
For unordered categories, use a test for categorical data, such as chi-square.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
A monotonic relationship
As one variable increases, the other consistently increases, or consistently decreases.
Why it matters
Spearman's rho measures how consistently the ranks rise or fall together; a U-shaped relationship can give a rho near zero.
How to check it
Inspect a scatterplot of the two variables.
Commonly used thresholds
No numeric threshold: judge the plot for a consistent direction.
If it is violated
A relationship that rises then falls is missed.
What to do instead
Describe the relationship in parts of the range, or model the curve directly.
Alternatives
- Kendall's tau-b: When there are many tied ranks or the sample is small.
Reporting example
Because [variable] was measured on an ordinal scale, Spearman's rank correlation was used; a scatterplot showed a monotonic relationship.
Illustrative numbers; replace them with your own.
Common mistakes
- Assuming Spearman needs no checks at all: it still assumes independent observations and a monotonic relationship.
References awaiting academic review.
Simple linear regression
Predicts a numeric outcome from one variable.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Linearity
The relationship between the variables is a straight line.
Why it matters
Pearson correlation and linear regression measure straight-line relationships; a curved one is underestimated or missed.
How to check it
- Inspect a scatterplot of each pair.
- In regression, plot residuals against predicted values: they should scatter evenly around zero with no curve.
Commonly used thresholds
- No numeric threshold: judge the plots.
- A clear curve in the residual plot means the relationship isn't linear.
If it is violated
The strength of the relationship is underestimated, and predictions are wrong in parts of the range.
What to do instead
- Add a squared term, or transform a variable.
- Use Spearman's correlation for a relationship that is monotonic but curved.
Normally distributed residuals
The residuals (the differences between observed and predicted values) are approximately normal.
Why it matters
Regression's significance tests and confidence intervals assume normal residuals; the predictors themselves needn't be normal.
How to check it
- Inspect a histogram and a normal P–P or Q–Q plot of the standardised residuals.
- Run the Shapiro–Wilk test on the residuals.
Commonly used thresholds
- Standardised residuals should roughly follow the diagonal of the P–P plot.
- Shapiro–Wilk p above .05 is consistent with normal residuals; with large samples, rely on plots.
If it is violated
Coefficients stay unbiased, but p-values and confidence intervals can be inaccurate, especially in small samples.
What to do instead
- Use bootstrapped confidence intervals.
- Transform the outcome.
- Use robust regression.
Homoscedasticity
The spread of the residuals is the same at every level of the predicted values.
Why it matters
Standard errors assume equal spread; unequal spread makes some coefficients look more or less precise than they are.
How to check it
- Plot standardised residuals against standardised predicted values: the band should have the same width throughout.
- Use the Breusch–Pagan or White test.
Commonly used thresholds
- Breusch–Pagan p above .05 is consistent with equal spread.
- A funnel shape in the residual plot suggests unequal spread.
If it is violated
Coefficients stay unbiased, but their standard errors and p-values are wrong.
What to do instead
- Use heteroscedasticity-consistent (robust) standard errors.
- Transform the outcome.
- Use weighted least squares.
Independent errors
The residuals of any two observations are unrelated, such as over time or in the order data were collected.
Why it matters
Related residuals make standard errors too small.
How to check it
- Run the Durbin–Watson test.
- Plot residuals in the order the data were collected.
Commonly used thresholds
Durbin–Watson values near 2 suggest no autocorrelation; values between about 1.5 and 2.5 are commonly treated as acceptable.
If it is violated
Significance tests become too liberal.
What to do instead
Model the time order, such as with time-series or multilevel methods.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Robust (heteroscedasticity-consistent) standard errors: When the residuals' spread is unequal.
- Bootstrapped confidence intervals: When the residuals aren't normal, especially in smaller samples.
- Transforming the outcome: When the outcome is strongly skewed or the relationship is curved.
Non-parametric alternatives
- Spearman's rank correlation: When only the strength of a monotonic relationship is needed, not a prediction equation.
Reporting example
The residual plot showed no pattern, the P–P plot showed approximately normal residuals, no standardised residual exceeded ±3, and the Durbin–Watson statistic was 1.94.
Illustrative numbers; replace them with your own.
Common mistakes
- Checking the normality of each variable instead of the residuals.
- Skipping the residual plots, which reveal non-linearity and unequal spread together.
- Deleting outliers without a reason other than their size, and without reporting it.
References awaiting academic review.
Multiple regression
Predicts a numeric outcome from several variables at once, each adjusted for the others.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Linearity
The relationship between the variables is a straight line.
Why it matters
Pearson correlation and linear regression measure straight-line relationships; a curved one is underestimated or missed.
How to check it
- Inspect a scatterplot of each pair.
- In regression, plot residuals against predicted values: they should scatter evenly around zero with no curve.
Commonly used thresholds
- No numeric threshold: judge the plots.
- A clear curve in the residual plot means the relationship isn't linear.
If it is violated
The strength of the relationship is underestimated, and predictions are wrong in parts of the range.
What to do instead
- Add a squared term, or transform a variable.
- Use Spearman's correlation for a relationship that is monotonic but curved.
Normally distributed residuals
The residuals (the differences between observed and predicted values) are approximately normal.
Why it matters
Regression's significance tests and confidence intervals assume normal residuals; the predictors themselves needn't be normal.
How to check it
- Inspect a histogram and a normal P–P or Q–Q plot of the standardised residuals.
- Run the Shapiro–Wilk test on the residuals.
Commonly used thresholds
- Standardised residuals should roughly follow the diagonal of the P–P plot.
- Shapiro–Wilk p above .05 is consistent with normal residuals; with large samples, rely on plots.
If it is violated
Coefficients stay unbiased, but p-values and confidence intervals can be inaccurate, especially in small samples.
What to do instead
- Use bootstrapped confidence intervals.
- Transform the outcome.
- Use robust regression.
Homoscedasticity
The spread of the residuals is the same at every level of the predicted values.
Why it matters
Standard errors assume equal spread; unequal spread makes some coefficients look more or less precise than they are.
How to check it
- Plot standardised residuals against standardised predicted values: the band should have the same width throughout.
- Use the Breusch–Pagan or White test.
Commonly used thresholds
- Breusch–Pagan p above .05 is consistent with equal spread.
- A funnel shape in the residual plot suggests unequal spread.
If it is violated
Coefficients stay unbiased, but their standard errors and p-values are wrong.
What to do instead
- Use heteroscedasticity-consistent (robust) standard errors.
- Transform the outcome.
- Use weighted least squares.
Independent errors
The residuals of any two observations are unrelated, such as over time or in the order data were collected.
Why it matters
Related residuals make standard errors too small.
How to check it
- Run the Durbin–Watson test.
- Plot residuals in the order the data were collected.
Commonly used thresholds
Durbin–Watson values near 2 suggest no autocorrelation; values between about 1.5 and 2.5 are commonly treated as acceptable.
If it is violated
Significance tests become too liberal.
What to do instead
Model the time order, such as with time-series or multilevel methods.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
No multicollinearity
The predictors aren't too highly correlated with each other.
Why it matters
Overlapping predictors compete to explain the same variation, so their individual coefficients become unstable and hard to interpret.
How to check it
- Inspect correlations between predictors.
- Check the variance inflation factor (VIF) and tolerance for each predictor.
Commonly used thresholds
- A VIF below 10 (tolerance above .10) is a commonly used lenient cut-off; below 5 (tolerance above .20) is stricter.
- Correlations between predictors above about .80 to .90 are a warning sign.
If it is violated
Coefficients and their signs can change unpredictably, and important predictors can look non-significant.
What to do instead
- Remove or combine overlapping predictors, if theory allows.
- Centre variables before creating interaction terms.
Alternatives
- Robust (heteroscedasticity-consistent) standard errors: When the residuals' spread is unequal.
- Bootstrapped confidence intervals: When the residuals aren't normal, especially in smaller samples.
- Transforming the outcome: When the outcome is strongly skewed or the relationship is curved.
Reporting example
VIF values ranged from 1.12 to 2.35, below the common threshold of 10; the residual plots showed no pattern, the residuals were approximately normal, and the Durbin–Watson statistic was 1.87.
Illustrative numbers; replace them with your own.
Common mistakes
- Checking the normality of each variable instead of the residuals.
- Skipping the residual plots, which reveal non-linearity and unequal spread together.
- Deleting outliers without a reason other than their size, and without reporting it.
- Reporting coefficients without checking multicollinearity first.
References awaiting academic review.
Hierarchical regression
Adds predictors in stages, showing what each set adds beyond the ones before, such as control variables first.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Linearity
The relationship between the variables is a straight line.
Why it matters
Pearson correlation and linear regression measure straight-line relationships; a curved one is underestimated or missed.
How to check it
- Inspect a scatterplot of each pair.
- In regression, plot residuals against predicted values: they should scatter evenly around zero with no curve.
Commonly used thresholds
- No numeric threshold: judge the plots.
- A clear curve in the residual plot means the relationship isn't linear.
If it is violated
The strength of the relationship is underestimated, and predictions are wrong in parts of the range.
What to do instead
- Add a squared term, or transform a variable.
- Use Spearman's correlation for a relationship that is monotonic but curved.
Normally distributed residuals
The residuals (the differences between observed and predicted values) are approximately normal.
Why it matters
Regression's significance tests and confidence intervals assume normal residuals; the predictors themselves needn't be normal.
How to check it
- Inspect a histogram and a normal P–P or Q–Q plot of the standardised residuals.
- Run the Shapiro–Wilk test on the residuals.
Commonly used thresholds
- Standardised residuals should roughly follow the diagonal of the P–P plot.
- Shapiro–Wilk p above .05 is consistent with normal residuals; with large samples, rely on plots.
If it is violated
Coefficients stay unbiased, but p-values and confidence intervals can be inaccurate, especially in small samples.
What to do instead
- Use bootstrapped confidence intervals.
- Transform the outcome.
- Use robust regression.
Homoscedasticity
The spread of the residuals is the same at every level of the predicted values.
Why it matters
Standard errors assume equal spread; unequal spread makes some coefficients look more or less precise than they are.
How to check it
- Plot standardised residuals against standardised predicted values: the band should have the same width throughout.
- Use the Breusch–Pagan or White test.
Commonly used thresholds
- Breusch–Pagan p above .05 is consistent with equal spread.
- A funnel shape in the residual plot suggests unequal spread.
If it is violated
Coefficients stay unbiased, but their standard errors and p-values are wrong.
What to do instead
- Use heteroscedasticity-consistent (robust) standard errors.
- Transform the outcome.
- Use weighted least squares.
Independent errors
The residuals of any two observations are unrelated, such as over time or in the order data were collected.
Why it matters
Related residuals make standard errors too small.
How to check it
- Run the Durbin–Watson test.
- Plot residuals in the order the data were collected.
Commonly used thresholds
Durbin–Watson values near 2 suggest no autocorrelation; values between about 1.5 and 2.5 are commonly treated as acceptable.
If it is violated
Significance tests become too liberal.
What to do instead
Model the time order, such as with time-series or multilevel methods.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
No multicollinearity
The predictors aren't too highly correlated with each other.
Why it matters
Overlapping predictors compete to explain the same variation, so their individual coefficients become unstable and hard to interpret.
How to check it
- Inspect correlations between predictors.
- Check the variance inflation factor (VIF) and tolerance for each predictor.
Commonly used thresholds
- A VIF below 10 (tolerance above .10) is a commonly used lenient cut-off; below 5 (tolerance above .20) is stricter.
- Correlations between predictors above about .80 to .90 are a warning sign.
If it is violated
Coefficients and their signs can change unpredictably, and important predictors can look non-significant.
What to do instead
- Remove or combine overlapping predictors, if theory allows.
- Centre variables before creating interaction terms.
Alternatives
- Robust (heteroscedasticity-consistent) standard errors: When the residuals' spread is unequal.
- Bootstrapped confidence intervals: When the residuals aren't normal, especially in smaller samples.
- Transforming the outcome: When the outcome is strongly skewed or the relationship is curved.
Reporting example
Assumptions were checked on the final model: VIF values were below 3, the residuals were approximately normal and evenly spread, and no case had a Cook's distance above 1.
Illustrative numbers; replace them with your own.
Common mistakes
- Checking the normality of each variable instead of the residuals.
- Skipping the residual plots, which reveal non-linearity and unequal spread together.
- Deleting outliers without a reason other than their size, and without reporting it.
- Checking assumptions only for the first step, when the final model is the one interpreted.
References awaiting academic review.
Logistic regression
Predicts the chance of a two-category outcome, such as yes or no.
Assumptions to check
A two-category outcome
The outcome has exactly two categories, such as yes and no.
Why it matters
Binary logistic regression models the chance of one of two outcomes.
How to check it
Check the outcome's categories.
Commonly used thresholds
Exactly two mutually exclusive categories.
If it is violated
An outcome with more categories doesn't fit binary logistic regression.
What to do instead
- Combine categories into two, if theory supports it.
- Use multinomial or ordinal logistic regression for three or more categories.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Linearity of the logit
Each numeric predictor has a straight-line relationship with the log odds of the outcome.
Why it matters
Logistic regression models the log odds as a straight line in each numeric predictor; a curved relationship is misrepresented.
How to check it
Use the Box–Tidwell test: add each numeric predictor multiplied by its natural log, and test those terms.
Commonly used thresholds
Non-significant Box–Tidwell terms are consistent with linearity; a Bonferroni-corrected level is often used.
If it is violated
Odds ratios for the affected predictor are misleading.
What to do instead
Transform the predictor, or group it into categories if theory supports it.
No multicollinearity
The predictors aren't too highly correlated with each other.
Why it matters
Overlapping predictors compete to explain the same variation, so their individual coefficients become unstable and hard to interpret.
How to check it
- Inspect correlations between predictors.
- Check the variance inflation factor (VIF) and tolerance for each predictor.
Commonly used thresholds
- A VIF below 10 (tolerance above .10) is a commonly used lenient cut-off; below 5 (tolerance above .20) is stricter.
- Correlations between predictors above about .80 to .90 are a warning sign.
If it is violated
Coefficients and their signs can change unpredictably, and important predictors can look non-significant.
What to do instead
- Remove or combine overlapping predictors, if theory allows.
- Centre variables before creating interaction terms.
Enough cases per predictor
The smaller outcome group has enough cases for the number of predictors.
Why it matters
With few cases in the smaller group, estimates become unstable or can't be computed.
How to check it
Count the cases in the smaller outcome category and divide by the number of predictors.
Commonly used thresholds
A common rule of thumb is at least 10 cases in the smaller outcome group per predictor; it is debated and some recommend more.
If it is violated
Odds ratios can be very large with very wide confidence intervals, or the model may fail to converge.
What to do instead
- Use fewer predictors.
- Use penalised (Firth) logistic regression.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Penalised (Firth) logistic regression: When events are few or a predictor perfectly separates the outcomes.
- Multinomial or ordinal logistic regression: When the outcome has more than two categories.
Non-parametric alternatives
- Chi-square test of independence: When the predictors are categorical and only an association is needed.
Reporting example
The Box–Tidwell procedure showed a linear relationship between each numeric predictor and the logit (all p > .05), VIF values were below 2, and the smaller outcome group had 14 cases per predictor.
Illustrative numbers; replace them with your own.
Common mistakes
- Testing normality, which logistic regression doesn't assume.
- Using many predictors with few cases in the smaller outcome group.
References awaiting academic review.
One-sample t-test
Compares the mean of a numeric outcome in one sample with a known or hypothesised value, such as a target or a published average.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Normality
The outcome is approximately normally distributed within each group.
Why it matters
Parametric tests use the normal distribution to judge how surprising a result is; strong non-normality, especially in small samples, makes p-values inaccurate.
How to check it
- Inspect histograms and Q–Q plots for each group.
- Check skewness and kurtosis.
- Run the Shapiro–Wilk test, bearing in mind its power depends on sample size.
Commonly used thresholds
- Shapiro–Wilk p above .05 is consistent with normality; with large samples it flags trivial departures, so rely on plots.
- Skewness and kurtosis between −2 and +2 are often treated as acceptable; some researchers use −1 to +1.
If it is violated
With small samples, p-values and confidence intervals can be wrong. With larger samples, tests of means are commonly considered robust to moderate non-normality.
What to do instead
- Use the non-parametric alternative.
- Transform the variable, such as a log transformation for right skew, and report it.
- Use bootstrapped confidence intervals.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Bootstrapped confidence interval for the mean: When the outcome isn't normal but the mean is still the value of interest.
Non-parametric alternatives
- One-sample Wilcoxon signed-rank test: When the outcome is ordinal, or not normal in a small sample; it compares the median with the stated value.
Reporting example
[Outcome] was approximately normal (Shapiro–Wilk p = .27), with no outliers beyond ±3 standard deviations, so a one-sample t-test compared the mean with the stated value of [value].
Illustrative numbers; replace them with your own.
Common mistakes
- Choosing the comparison value after seeing the data.
- Using it to compare two groups, which needs an independent-samples or paired-samples test.
References awaiting academic review.
Independent-samples t-test
Compares the means of a numeric outcome between two separate groups.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Normality
The outcome is approximately normally distributed within each group.
Why it matters
Parametric tests use the normal distribution to judge how surprising a result is; strong non-normality, especially in small samples, makes p-values inaccurate.
How to check it
- Inspect histograms and Q–Q plots for each group.
- Check skewness and kurtosis.
- Run the Shapiro–Wilk test, bearing in mind its power depends on sample size.
Commonly used thresholds
- Shapiro–Wilk p above .05 is consistent with normality; with large samples it flags trivial departures, so rely on plots.
- Skewness and kurtosis between −2 and +2 are often treated as acceptable; some researchers use −1 to +1.
If it is violated
With small samples, p-values and confidence intervals can be wrong. With larger samples, tests of means are commonly considered robust to moderate non-normality.
What to do instead
- Use the non-parametric alternative.
- Transform the variable, such as a log transformation for right skew, and report it.
- Use bootstrapped confidence intervals.
Homogeneity of variance
The groups being compared have similar variances.
Why it matters
Tests that pool the groups' variances give the wrong answer when one group varies much more than another, especially with unequal group sizes.
How to check it
- Run Levene's test.
- Compare the largest and smallest group variances.
Commonly used thresholds
- Levene's p above .05 is consistent with equal variances.
- A ratio of largest to smallest variance below about 2 to 3 is a common rule of thumb, especially when groups are similar in size.
If it is violated
p-values become inaccurate, particularly when the smaller group has the larger variance.
What to do instead
- Use Welch's t-test, or Welch's ANOVA with Games–Howell post hoc tests.
- Report Levene's test and the correction used.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Welch's t-test: When the groups' variances differ; many recommend it by default.
Non-parametric alternatives
- Mann–Whitney U test: When the outcome is ordinal, or not normal in small groups.
Reporting example
[Outcome] was approximately normal in both groups (Shapiro–Wilk p = .21 and p = .34), and Levene's test indicated equal variances, F(1, 118) = 0.92, p = .339.
Illustrative numbers; replace them with your own.
Common mistakes
- Checking normality for the whole sample instead of within each group.
- Ignoring Levene's test, or not saying which version of the t-test was used.
References awaiting academic review.
Paired-samples t-test
Compares the means of a numeric outcome measured twice on the same participants, such as before and after.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Paired observations
Each measurement in one condition is matched to one in the other, usually the same participant measured twice.
Why it matters
Paired tests analyse the difference within each pair, which removes differences between people.
How to check it
Check the design: are the same participants, or matched pairs, measured in both conditions?
Commonly used thresholds
Every participant has a measurement at both times or in both conditions.
If it is violated
Treating independent groups as paired, or paired data as independent, gives the wrong standard error.
What to do instead
- Use the independent-samples version for separate groups.
- Use repeated measures methods for three or more times.
Normally distributed differences
The differences between each pair of measurements are approximately normal.
Why it matters
The paired t-test is a one-sample test of the differences, so it is their distribution that matters, not each measurement's.
How to check it
- Compute each participant's difference and inspect its histogram and Q–Q plot.
- Run the Shapiro–Wilk test on the differences.
Commonly used thresholds
- Shapiro–Wilk p above .05 on the differences is consistent with normality.
- Skewness and kurtosis of the differences between −2 and +2 are often treated as acceptable.
If it is violated
With few pairs, p-values can be inaccurate.
What to do instead
- Use the Wilcoxon signed-rank test.
- Use a bootstrapped confidence interval for the mean difference.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Bootstrapped confidence interval for the mean difference: When the differences aren't normal but a mean difference is still wanted.
Non-parametric alternatives
- Wilcoxon signed-rank test: When the differences aren't normal, or the outcome is ordinal.
Reporting example
The differences between the two measurements were approximately normal (Shapiro–Wilk p = .48), with no outliers beyond ±3 standard deviations.
Illustrative numbers; replace them with your own.
Common mistakes
- Checking normality of each time point instead of the differences.
- Using it for two separate groups.
References awaiting academic review.
One-way ANOVA
Compares the means of a numeric outcome across three or more independent groups.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Normality
The outcome is approximately normally distributed within each group.
Why it matters
Parametric tests use the normal distribution to judge how surprising a result is; strong non-normality, especially in small samples, makes p-values inaccurate.
How to check it
- Inspect histograms and Q–Q plots for each group.
- Check skewness and kurtosis.
- Run the Shapiro–Wilk test, bearing in mind its power depends on sample size.
Commonly used thresholds
- Shapiro–Wilk p above .05 is consistent with normality; with large samples it flags trivial departures, so rely on plots.
- Skewness and kurtosis between −2 and +2 are often treated as acceptable; some researchers use −1 to +1.
If it is violated
With small samples, p-values and confidence intervals can be wrong. With larger samples, tests of means are commonly considered robust to moderate non-normality.
What to do instead
- Use the non-parametric alternative.
- Transform the variable, such as a log transformation for right skew, and report it.
- Use bootstrapped confidence intervals.
Homogeneity of variance
The groups being compared have similar variances.
Why it matters
Tests that pool the groups' variances give the wrong answer when one group varies much more than another, especially with unequal group sizes.
How to check it
- Run Levene's test.
- Compare the largest and smallest group variances.
Commonly used thresholds
- Levene's p above .05 is consistent with equal variances.
- A ratio of largest to smallest variance below about 2 to 3 is a common rule of thumb, especially when groups are similar in size.
If it is violated
p-values become inaccurate, particularly when the smaller group has the larger variance.
What to do instead
- Use Welch's t-test, or Welch's ANOVA with Games–Howell post hoc tests.
- Report Levene's test and the correction used.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Adequate group sizes
Each group, or combination of groups, has enough participants.
Why it matters
Small or very unequal groups make tests less robust to other violations, and make some effects impossible to estimate.
How to check it
Count the participants in every group or cell.
Commonly used thresholds
- Roughly equal groups make most tests more robust.
- In MANOVA, each cell should have more cases than there are outcome variables.
If it is violated
Tests become sensitive to unequal variances, and power falls.
What to do instead
- Plan equal groups where possible.
- Use Welch-type corrections, or combine small groups if theory allows.
Alternatives
- Welch's ANOVA with Games–Howell post hoc tests: When the groups' variances differ.
Non-parametric alternatives
- Kruskal–Wallis test: When the outcome is ordinal, or not normal in small groups.
Reporting example
[Outcome] was approximately normal in each group, and Levene's test indicated equal variances, F(2, 117) = 1.08, p = .343.
Illustrative numbers; replace them with your own.
Common mistakes
- Checking normality for the whole sample instead of within each group.
- Using standard post hoc tests after finding unequal variances.
References awaiting academic review.
Two-way ANOVA
Compares means across two grouping factors at once, including whether they interact.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Normality
The outcome is approximately normally distributed within each group.
Why it matters
Parametric tests use the normal distribution to judge how surprising a result is; strong non-normality, especially in small samples, makes p-values inaccurate.
How to check it
- Inspect histograms and Q–Q plots for each group.
- Check skewness and kurtosis.
- Run the Shapiro–Wilk test, bearing in mind its power depends on sample size.
Commonly used thresholds
- Shapiro–Wilk p above .05 is consistent with normality; with large samples it flags trivial departures, so rely on plots.
- Skewness and kurtosis between −2 and +2 are often treated as acceptable; some researchers use −1 to +1.
If it is violated
With small samples, p-values and confidence intervals can be wrong. With larger samples, tests of means are commonly considered robust to moderate non-normality.
What to do instead
- Use the non-parametric alternative.
- Transform the variable, such as a log transformation for right skew, and report it.
- Use bootstrapped confidence intervals.
Homogeneity of variance
The groups being compared have similar variances.
Why it matters
Tests that pool the groups' variances give the wrong answer when one group varies much more than another, especially with unequal group sizes.
How to check it
- Run Levene's test.
- Compare the largest and smallest group variances.
Commonly used thresholds
- Levene's p above .05 is consistent with equal variances.
- A ratio of largest to smallest variance below about 2 to 3 is a common rule of thumb, especially when groups are similar in size.
If it is violated
p-values become inaccurate, particularly when the smaller group has the larger variance.
What to do instead
- Use Welch's t-test, or Welch's ANOVA with Games–Howell post hoc tests.
- Report Levene's test and the correction used.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Adequate group sizes
Each group, or combination of groups, has enough participants.
Why it matters
Small or very unequal groups make tests less robust to other violations, and make some effects impossible to estimate.
How to check it
Count the participants in every group or cell.
Commonly used thresholds
- Roughly equal groups make most tests more robust.
- In MANOVA, each cell should have more cases than there are outcome variables.
If it is violated
Tests become sensitive to unequal variances, and power falls.
What to do instead
- Plan equal groups where possible.
- Use Welch-type corrections, or combine small groups if theory allows.
Alternatives
- Robust or bootstrapped ANOVA: When normality or equal variances clearly fail.
- Transforming the outcome: When the outcome is strongly skewed.
Non-parametric alternatives
- Aligned rank transform ANOVA: When the outcome is ordinal or not normal and the interaction must still be tested.
Reporting example
Levene's test indicated equal variances across the six cells, F(5, 114) = 1.21, p = .308, and residuals were approximately normal.
Illustrative numbers; replace them with your own.
Common mistakes
- Checking assumptions for each factor separately instead of for every cell.
- Ignoring very unequal cell sizes.
References awaiting academic review.
Repeated measures ANOVA
Compares the means of a numeric outcome measured three or more times on the same participants.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Paired observations
Each measurement in one condition is matched to one in the other, usually the same participant measured twice.
Why it matters
Paired tests analyse the difference within each pair, which removes differences between people.
How to check it
Check the design: are the same participants, or matched pairs, measured in both conditions?
Commonly used thresholds
Every participant has a measurement at both times or in both conditions.
If it is violated
Treating independent groups as paired, or paired data as independent, gives the wrong standard error.
What to do instead
- Use the independent-samples version for separate groups.
- Use repeated measures methods for three or more times.
Normality
The outcome is approximately normally distributed within each group.
Why it matters
Parametric tests use the normal distribution to judge how surprising a result is; strong non-normality, especially in small samples, makes p-values inaccurate.
How to check it
- Inspect histograms and Q–Q plots for each group.
- Check skewness and kurtosis.
- Run the Shapiro–Wilk test, bearing in mind its power depends on sample size.
Commonly used thresholds
- Shapiro–Wilk p above .05 is consistent with normality; with large samples it flags trivial departures, so rely on plots.
- Skewness and kurtosis between −2 and +2 are often treated as acceptable; some researchers use −1 to +1.
If it is violated
With small samples, p-values and confidence intervals can be wrong. With larger samples, tests of means are commonly considered robust to moderate non-normality.
What to do instead
- Use the non-parametric alternative.
- Transform the variable, such as a log transformation for right skew, and report it.
- Use bootstrapped confidence intervals.
Sphericity
The variances of the differences between every pair of time points are similar.
Why it matters
Repeated measures ANOVA's F test assumes it; when it fails, F is too large and false positives increase.
How to check it
- Run Mauchly's test.
- Look at the Greenhouse–Geisser epsilon.
Commonly used thresholds
- Mauchly's p above .05 is consistent with sphericity; it only applies with three or more time points.
- If sphericity fails, use the Greenhouse–Geisser correction when epsilon is below about .75, and Huynh–Feldt when it is above.
If it is violated
The F test is too liberal: it finds effects that aren't there more often than the stated level.
What to do instead
- Apply the Greenhouse–Geisser or Huynh–Feldt correction.
- Use a multivariate approach or a linear mixed model.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Greenhouse–Geisser or Huynh–Feldt correction: When sphericity fails.
- Linear mixed model: When some participants miss a time point, or times are unevenly spaced.
Non-parametric alternatives
- Friedman test: When the outcome is ordinal, or not normal with few participants.
Reporting example
Mauchly's test indicated that sphericity was violated, χ²(2) = 8.41, p = .015, so the Greenhouse–Geisser correction was applied (ε = .68).
Illustrative numbers; replace them with your own.
Common mistakes
- Reporting uncorrected F after Mauchly's test is significant.
- Dropping participants with any missing time point without saying so.
References awaiting academic review.
ANCOVA
Compares group means on a numeric outcome after adjusting for a numeric covariate, such as a control variable or a pretest score.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Normally distributed residuals
The residuals (the differences between observed and predicted values) are approximately normal.
Why it matters
Regression's significance tests and confidence intervals assume normal residuals; the predictors themselves needn't be normal.
How to check it
- Inspect a histogram and a normal P–P or Q–Q plot of the standardised residuals.
- Run the Shapiro–Wilk test on the residuals.
Commonly used thresholds
- Standardised residuals should roughly follow the diagonal of the P–P plot.
- Shapiro–Wilk p above .05 is consistent with normal residuals; with large samples, rely on plots.
If it is violated
Coefficients stay unbiased, but p-values and confidence intervals can be inaccurate, especially in small samples.
What to do instead
- Use bootstrapped confidence intervals.
- Transform the outcome.
- Use robust regression.
Homogeneity of variance
The groups being compared have similar variances.
Why it matters
Tests that pool the groups' variances give the wrong answer when one group varies much more than another, especially with unequal group sizes.
How to check it
- Run Levene's test.
- Compare the largest and smallest group variances.
Commonly used thresholds
- Levene's p above .05 is consistent with equal variances.
- A ratio of largest to smallest variance below about 2 to 3 is a common rule of thumb, especially when groups are similar in size.
If it is violated
p-values become inaccurate, particularly when the smaller group has the larger variance.
What to do instead
- Use Welch's t-test, or Welch's ANOVA with Games–Howell post hoc tests.
- Report Levene's test and the correction used.
Linearity
The relationship between the variables is a straight line.
Why it matters
Pearson correlation and linear regression measure straight-line relationships; a curved one is underestimated or missed.
How to check it
- Inspect a scatterplot of each pair.
- In regression, plot residuals against predicted values: they should scatter evenly around zero with no curve.
Commonly used thresholds
- No numeric threshold: judge the plots.
- A clear curve in the residual plot means the relationship isn't linear.
If it is violated
The strength of the relationship is underestimated, and predictions are wrong in parts of the range.
What to do instead
- Add a squared term, or transform a variable.
- Use Spearman's correlation for a relationship that is monotonic but curved.
Homogeneity of regression slopes
The covariate relates to the outcome in the same way in every group.
Why it matters
ANCOVA adjusts every group by one common slope; if the slopes differ, the adjustment is wrong for some groups.
How to check it
Add a group × covariate interaction to the model and test it.
Commonly used thresholds
A non-significant interaction (p above .05) is consistent with equal slopes.
If it is violated
The adjusted group means can mislead, because the group difference depends on the covariate's value.
What to do instead
- Report the interaction instead, using regression with the interaction term.
- Describe the group difference at different covariate values.
Covariate independent of the grouping
The covariate isn't affected by group membership, usually because it is measured before the intervention.
Why it matters
Adjusting for a covariate that the groups themselves changed can remove part of the effect being tested.
How to check it
- Check when the covariate was measured.
- Compare the covariate across groups: in a randomised design, differences should be small.
Commonly used thresholds
No numeric threshold: it follows from the design and timing of measurement.
If it is violated
The adjusted effect can be biased, towards zero or away from it.
What to do instead
- Measure covariates before the intervention.
- Don't adjust for variables the treatment may change.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Regression with a group × covariate interaction: When the regression slopes differ between groups.
Non-parametric alternatives
- Quade's rank analysis of covariance: When the outcome is ordinal or clearly not normal.
Reporting example
The group × covariate interaction was not significant, F(1, 116) = 0.74, p = .391, indicating homogeneous regression slopes, and the covariate was linearly related to the outcome in each group.
Illustrative numbers; replace them with your own.
Common mistakes
- Skipping the test of homogeneous slopes.
- Adjusting for a covariate measured after the intervention.
References awaiting academic review.
MANOVA
Compares groups on several related numeric outcomes at once.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Multivariate normality
The variables are jointly normally distributed, not just each one on its own.
Why it matters
MANOVA and maximum likelihood SEM rely on it for their test statistics and standard errors.
How to check it
- Check each variable's normality first.
- Use Mardia's test of multivariate skewness and kurtosis.
- Check for multivariate outliers with Mahalanobis distance.
Commonly used thresholds
Mardia's coefficients non-significant, or a normalised multivariate kurtosis below about 5, are commonly treated as acceptable.
If it is violated
Chi-square tests of fit become too large and standard errors too small, so models look worse and effects look stronger than they are.
What to do instead
- Use robust estimation, such as maximum likelihood with Satorra–Bentler correction.
- Use bootstrapping.
- In MANOVA, report Pillai's trace, which is the most robust statistic.
Homogeneity of covariance matrices
The groups have similar variances and similar relationships between the outcomes.
Why it matters
MANOVA pools the groups' covariance matrices; very different matrices distort its test.
How to check it
Run Box's M test.
Commonly used thresholds
Box's M is very sensitive, so a strict level is commonly used: p above .001 is treated as acceptable.
If it is violated
The multivariate test becomes inaccurate, especially with unequal group sizes.
What to do instead
- Report Pillai's trace, which is the most robust multivariate statistic.
- Keep group sizes equal where possible.
Moderately related outcomes
The outcome variables are related to each other, but not so strongly that they measure the same thing.
Why it matters
MANOVA gains from analysing related outcomes together; unrelated outcomes are better analysed separately, and near-identical ones are redundant.
How to check it
Inspect the correlations between the outcome variables.
Commonly used thresholds
Moderate correlations are ideal; correlations above about .80 to .90 suggest redundancy.
If it is violated
With unrelated outcomes MANOVA loses power; with near-identical ones it becomes unstable.
What to do instead
- Analyse unrelated outcomes with separate ANOVAs and a corrected significance level.
- Combine near-identical outcomes.
No multivariate outliers
No case has an unusual combination of values across several variables.
Why it matters
A case can look ordinary on each variable but unusual in combination, and still distort multivariate results.
How to check it
Compute Mahalanobis distance for each case.
Commonly used thresholds
A Mahalanobis distance beyond the chi-square critical value at p < .001, with degrees of freedom equal to the number of variables, is commonly flagged.
If it is violated
Multivariate estimates and fit statistics can be distorted.
What to do instead
- Check the case for errors.
- Report results with and without it.
Adequate group sizes
Each group, or combination of groups, has enough participants.
Why it matters
Small or very unequal groups make tests less robust to other violations, and make some effects impossible to estimate.
How to check it
Count the participants in every group or cell.
Commonly used thresholds
- Roughly equal groups make most tests more robust.
- In MANOVA, each cell should have more cases than there are outcome variables.
If it is violated
Tests become sensitive to unequal variances, and power falls.
What to do instead
- Plan equal groups where possible.
- Use Welch-type corrections, or combine small groups if theory allows.
Alternatives
- Pillai's trace: When assumptions are doubtful or groups are unequal; it is the most robust multivariate statistic.
- Separate ANOVAs with a corrected significance level: When the outcomes are unrelated.
Non-parametric alternatives
- Permutation MANOVA (PERMANOVA): When multivariate normality clearly fails.
Reporting example
Box's M test was not significant at the .001 level, M = 18.42, p = .042, and no multivariate outliers were found using Mahalanobis distance (p < .001 criterion).
Illustrative numbers; replace them with your own.
Common mistakes
- Treating Box's M as failed at p below .05, when .001 is the usual level.
- Including outcomes that are unrelated or nearly identical.
References awaiting academic review.
Chi-square test of independence
Tests whether two categorical variables are associated.
Assumptions to check
Categorical variables
Both variables are measured in categories.
Why it matters
The chi-square test compares counts in the cells of a table; it needs categories to count.
How to check it
Check each variable's measurement level and answer options.
Commonly used thresholds
Categories should be mutually exclusive: each participant falls in exactly one.
If it is violated
Numeric variables cut into categories lose information, and the cut-points can change the result.
What to do instead
Keep numeric variables numeric and use correlation, regression or a t-test instead.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Adequate expected counts
The counts expected in each cell, if the variables were unrelated, are large enough.
Why it matters
The chi-square test is an approximation that is poor when expected counts are small.
How to check it
Look at the expected counts in your software's output.
Commonly used thresholds
- At least 80% of cells should have expected counts of 5 or more, and none below 1.
- For a 2 × 2 table, every expected count should be 5 or more.
If it is violated
The p-value can be inaccurate.
What to do instead
- Use Fisher's exact test.
- Combine categories with few cases, if theory supports it.
Alternatives
- Combining sparse categories: When some categories have very few cases and theory allows merging them.
Non-parametric alternatives
- Fisher's exact test: When expected counts are too small for chi-square.
Reporting example
All expected counts were 5 or more, so the chi-square test of independence was used.
Illustrative numbers; replace them with your own.
Common mistakes
- Counting the same participant in more than one cell.
- Not checking the expected counts, which your software reports.
References awaiting academic review.
Factor analysis
Finds or confirms groups of items that measure the same underlying construct.
Assumptions to check
Factorability
The items share enough common variance for factors to be meaningful.
Why it matters
Factor analysis looks for groups of correlated items; if items are barely correlated, any factors found are unstable.
How to check it
- Check the Kaiser–Meyer–Olkin (KMO) measure.
- Run Bartlett's test of sphericity.
- Inspect the correlation matrix.
Commonly used thresholds
- KMO of .60 or more is commonly required; Kaiser treated .50 as the minimum.
- Bartlett's test should be significant (p below .05).
- Many correlations between items should be above about .30.
If it is violated
Factors may be unstable or uninterpretable.
What to do instead
- Remove items with low correlations or low individual KMO values.
- Collect more data.
Adequate sample for factor analysis
There are enough participants for a stable factor solution.
Why it matters
Correlations, and so factors, are unstable in small samples.
How to check it
Compare the sample size with the number of items and the strength of the loadings.
Commonly used thresholds
- Common guidance ranges from at least 100 participants to 300 or more.
- A ratio of 5 to 10 participants per item is often cited; strong loadings allow smaller samples.
If it is violated
Factors may not replicate in another sample.
What to do instead
- Collect more data.
- Use fewer items.
- Rely on established measures' published structure.
Linearity
The relationship between the variables is a straight line.
Why it matters
Pearson correlation and linear regression measure straight-line relationships; a curved one is underestimated or missed.
How to check it
- Inspect a scatterplot of each pair.
- In regression, plot residuals against predicted values: they should scatter evenly around zero with no curve.
Commonly used thresholds
- No numeric threshold: judge the plots.
- A clear curve in the residual plot means the relationship isn't linear.
If it is violated
The strength of the relationship is underestimated, and predictions are wrong in parts of the range.
What to do instead
- Add a squared term, or transform a variable.
- Use Spearman's correlation for a relationship that is monotonic but curved.
No influential outliers
No single case distorts the result.
Why it matters
Means, correlations and regression lines are pulled towards extreme values, so one case can change a conclusion.
How to check it
- Inspect boxplots and scatterplots.
- Check standardised residuals and Cook's distance in regression.
Commonly used thresholds
- Standardised scores or residuals beyond ±3 are commonly flagged; beyond ±2.5 in small samples.
- Cook's distance above 1 suggests an influential case; some use 4 divided by N.
If it is violated
Estimates and p-values can depend on one or two cases.
What to do instead
- Check whether the value is an error, and correct it if so.
- Report the analysis with and without the case.
- Use a rank-based or robust method.
Alternatives
- Principal axis factoring: When the items aren't normal enough for maximum likelihood extraction.
- Factor analysis of polychoric correlations: When the items are ordinal, such as single rating items with few points.
Reporting example
The Kaiser–Meyer–Olkin measure was .84, and Bartlett's test of sphericity was significant, χ²(66) = 812.45, p < .001, indicating the data were suitable for factor analysis.
Illustrative numbers; replace them with your own.
Common mistakes
- Skipping KMO and Bartlett's test.
- Running factor analysis on too few participants for the number of items.
References awaiting academic review.
Structural equation modelling
Tests a whole model of relationships at once, often between constructs measured by several items each.
Assumptions to check
Interval or ratio measurement
The outcome is measured on a numeric scale with equal intervals between values.
Why it matters
Means, standard deviations and the tests built on them assume the distance between 3 and 4 equals the distance between 4 and 5.
How to check it
- Check how each variable was measured: its measurement level and response format.
- For scale scores made from several rating items, state that the total or average is treated as interval.
Commonly used thresholds
- No numeric threshold: it is a question of measurement design.
- Single rating items are ordinal; scale scores from several items are commonly treated as interval, which should be stated.
If it is violated
Means and tests can mislead when categories aren't evenly spaced.
What to do instead
- Use a rank-based (non-parametric) test.
- Combine several rating items into a scale score, and report its reliability.
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
Multivariate normality
The variables are jointly normally distributed, not just each one on its own.
Why it matters
MANOVA and maximum likelihood SEM rely on it for their test statistics and standard errors.
How to check it
- Check each variable's normality first.
- Use Mardia's test of multivariate skewness and kurtosis.
- Check for multivariate outliers with Mahalanobis distance.
Commonly used thresholds
Mardia's coefficients non-significant, or a normalised multivariate kurtosis below about 5, are commonly treated as acceptable.
If it is violated
Chi-square tests of fit become too large and standard errors too small, so models look worse and effects look stronger than they are.
What to do instead
- Use robust estimation, such as maximum likelihood with Satorra–Bentler correction.
- Use bootstrapping.
- In MANOVA, report Pillai's trace, which is the most robust statistic.
No multivariate outliers
No case has an unusual combination of values across several variables.
Why it matters
A case can look ordinary on each variable but unusual in combination, and still distort multivariate results.
How to check it
Compute Mahalanobis distance for each case.
Commonly used thresholds
A Mahalanobis distance beyond the chi-square critical value at p < .001, with degrees of freedom equal to the number of variables, is commonly flagged.
If it is violated
Multivariate estimates and fit statistics can be distorted.
What to do instead
- Check the case for errors.
- Report results with and without it.
Model identification
The model has enough information to estimate every parameter.
Why it matters
An unidentified model has no unique solution, so its estimates are meaningless.
How to check it
- Check that degrees of freedom are zero or more.
- Check each latent construct has enough indicators.
Commonly used thresholds
- Each latent construct commonly needs at least three indicators, or two with constraints.
- The model's degrees of freedom must be zero or positive.
If it is violated
The software fails to estimate the model, or gives meaningless estimates.
What to do instead
- Add indicators.
- Fix or constrain parameters as theory allows.
Adequate sample for SEM
There are enough participants for the model's complexity.
Why it matters
CB-SEM relies on large-sample theory; small samples give unstable estimates, poor fit and failed estimation.
How to check it
Compare the sample with the number of estimated parameters.
Commonly used thresholds
- 200 or more is a commonly cited minimum for CB-SEM; ratios of 5 to 10 participants per estimated parameter are also used.
- For PLS-SEM, a common minimum is ten times the largest number of arrows pointing at one construct; power analysis is preferred.
If it is violated
Estimates are unstable, fit statistics are unreliable, and the model may not converge.
What to do instead
- Simplify the model.
- Use PLS-SEM for smaller samples, if its approach suits your aims.
A reliable and valid measurement model
Each construct's indicators load strongly on it, reliably and with convergent validity.
Why it matters
Paths between constructs are only as trustworthy as the measurement of the constructs.
How to check it
- Check indicator loadings.
- Check composite reliability and average variance extracted (AVE).
Commonly used thresholds
- Indicator loadings of .70 or more are commonly sought; .40 to .70 may be kept if reliability and AVE are adequate.
- Composite reliability between .70 and .95.
- AVE of .50 or more.
If it is violated
Weak measurement weakens and biases the paths between constructs.
What to do instead
- Remove weak indicators if content validity allows.
- Revise the measure before the main study.
Discriminant validity
Each construct is distinct from the others.
Why it matters
If two constructs are indistinguishable, paths between them are meaningless.
How to check it
- Check the heterotrait–monotrait ratio (HTMT).
- Compare each construct's square root of AVE with its correlations (Fornell–Larcker criterion).
Commonly used thresholds
- HTMT below .85 is strict; below .90 is commonly accepted for conceptually similar constructs.
- Each construct's square root of AVE should exceed its correlations with the others.
If it is violated
Constructs may be measuring the same thing, so relationships between them are inflated.
What to do instead
- Revise overlapping items.
- Merge constructs if theory supports it.
Alternatives
- Robust maximum likelihood (Satorra–Bentler): When multivariate normality fails.
- Weighted least squares estimation (such as WLSMV): When indicators are ordinal, such as rating items with few points.
Non-parametric alternatives
- PLS-SEM: When distributional assumptions or sample size are a concern and the aim is prediction.
Reporting example
Mardia's normalised multivariate kurtosis was 3.2, below the common threshold of 5, so maximum likelihood estimation was used; all standardised loadings exceeded .70.
Illustrative numbers; replace them with your own.
Common mistakes
- Assessing the structural model before confirming the measurement model.
- Using maximum likelihood with clearly non-normal or ordinal indicators without correction.
References awaiting academic review.
PLS-SEM
Estimates a structural model by maximising the variance explained in the outcomes.
Assumptions to check
Independence of observations
Each participant's data are unrelated to every other participant's, and each participant contributes once.
Why it matters
Tests assume every observation adds new information. Related observations make standard errors too small and p-values too low.
How to check it
- Check the design: are any participants measured more than once, or grouped in ways that make them alike, such as classes or families?
- Check the sampling technique: cluster sampling groups participants.
Commonly used thresholds
No numeric threshold: it follows from the design and data collection.
If it is violated
Violations inflate the chance of a false positive, often seriously.
What to do instead
- Use paired or repeated measures methods when the same people are measured more than once.
- Use multilevel models, or adjust for clustering, when participants are grouped.
A reliable and valid measurement model
Each construct's indicators load strongly on it, reliably and with convergent validity.
Why it matters
Paths between constructs are only as trustworthy as the measurement of the constructs.
How to check it
- Check indicator loadings.
- Check composite reliability and average variance extracted (AVE).
Commonly used thresholds
- Indicator loadings of .70 or more are commonly sought; .40 to .70 may be kept if reliability and AVE are adequate.
- Composite reliability between .70 and .95.
- AVE of .50 or more.
If it is violated
Weak measurement weakens and biases the paths between constructs.
What to do instead
- Remove weak indicators if content validity allows.
- Revise the measure before the main study.
Discriminant validity
Each construct is distinct from the others.
Why it matters
If two constructs are indistinguishable, paths between them are meaningless.
How to check it
- Check the heterotrait–monotrait ratio (HTMT).
- Compare each construct's square root of AVE with its correlations (Fornell–Larcker criterion).
Commonly used thresholds
- HTMT below .85 is strict; below .90 is commonly accepted for conceptually similar constructs.
- Each construct's square root of AVE should exceed its correlations with the others.
If it is violated
Constructs may be measuring the same thing, so relationships between them are inflated.
What to do instead
- Revise overlapping items.
- Merge constructs if theory supports it.
No collinearity between predictor constructs
Constructs predicting the same outcome aren't too highly correlated.
Why it matters
Overlapping predictor constructs make path coefficients unstable.
How to check it
Check the variance inflation factor (VIF) of predictor constructs in the structural model.
Commonly used thresholds
VIF below 3 is ideal in PLS-SEM guidance; below 5 is commonly accepted.
If it is violated
Path coefficients can be biased or change sign.
What to do instead
- Merge or remove overlapping constructs, if theory allows.
- Create a higher-order construct.
Adequate sample for SEM
There are enough participants for the model's complexity.
Why it matters
CB-SEM relies on large-sample theory; small samples give unstable estimates, poor fit and failed estimation.
How to check it
Compare the sample with the number of estimated parameters.
Commonly used thresholds
- 200 or more is a commonly cited minimum for CB-SEM; ratios of 5 to 10 participants per estimated parameter are also used.
- For PLS-SEM, a common minimum is ten times the largest number of arrows pointing at one construct; power analysis is preferred.
If it is violated
Estimates are unstable, fit statistics are unreliable, and the model may not converge.
What to do instead
- Simplify the model.
- Use PLS-SEM for smaller samples, if its approach suits your aims.
Alternatives
- CB-SEM: When the aim is confirming an established theory and its assumptions and sample size are met.
Reporting example
All indicator loadings exceeded .708, composite reliability ranged from .82 to .91, AVE exceeded .50, HTMT values were below .85, and structural VIF values were below 3.
Illustrative numbers; replace them with your own.
Common mistakes
- Skipping the measurement model assessment before interpreting paths.
- Believing PLS-SEM has no assumptions: independence and an adequate sample still matter.
References awaiting academic review.
Next: Results interpretation
What your results mean for your hypotheses, question and objectives.
What should I do next?
Next in the research journey: writing.
Word Counter
Counts words, characters, sentences and paragraphs, and estimates reading and speaking time.
Readability Checker
Estimates how hard your text is to read with five established formulas, and explains what each one measures.
Paragraph Counter
Counts paragraphs and shows how many words each has, with the average, shortest and longest.
How to structure an academic essay
How to build an academic essay around an argument: analysing the question, writing a thesis statement, planning the body, writing introductions and conclusions, common structures for different essay types, and a worked plan.
How the checker works
- The checker reads your project draft: variables and how they are measured, hypotheses, research design, sampling, sample size, and the analyses the Data Analysis Recommender plans from them. Enter them in the first step; later versions will bring them across from the other research tools.
- For each planned analysis, it lists the assumptions to check. What your project already shows, such as measurement levels, repeated measurement, clustering and planned sample size, is judged now; the rest is marked for you to review once data are collected.
- Every assumption explains why it matters, how researchers usually check it, the thresholds commonly used, and what to do if it fails, including non-parametric alternatives.
- Nothing is scored. Each assumption is marked Looks aligned, Worth checking, Needs clarification, Missing or For you to review.
What this tool can't do
- The checker doesn't test assumptions or look at data. It explains what to check, and judges only what your project draft already shows, such as measurement levels, repeated measurement, clustering and planned sample size.
- Thresholds are conventions that differ between textbooks and fields; the ones given are commonly used, not fixed rules.
- Formal tests such as Shapiro–Wilk and Levene's depend on sample size: large samples flag trivial departures and small samples miss real ones. Use plots alongside them.
- Analyses come from the plan the Data Analysis Recommender makes from your project; if the project is incomplete, so is the checklist.
- Reporting examples use illustrative numbers; replace them with your own results.
Awaiting review
- Statistical review: every assumption, threshold, remedy and alternative, by a statistician.
- Threshold review: the conventions cited, which vary between textbooks.
- References: sources for each assumption and threshold, after academic review.
- Reporting style review: the reporting examples against APA and other common styles.
Privacy
Everything stays in your browser. Nothing is sent anywhere or saved, and it is cleared when you leave the page. If you start a project in your research workspace, your work is also saved in this browser, and only this browser, until you delete the project.