Levels of Measurement, Choosing Statistical Tests, Sign Test & Type I/II Errors

1

Levels of Measurement: Nominal, Ordinal, and Interval Data

Before choosing a statistical test, you must classify the level of measurement of the data: 1. Nominal data (categorical/frequency count): data consists of discrete categories where each participant falls into one box and you simply count frequencies (e.g., number of people who 'obeyed' vs 'defied', or 'secure' vs 'insecure-avoidant' attachment). 2. Ordinal data (ranked/subjective scale): data can be placed in rank order from lowest to highest, but the intervals between points are not equal or objective (e.g., self-report rating scales from 1-10 for anxiety or happiness, or psychological test/memory scores where jumping from 4 to 5 words is not guaranteed to be the exact same mental interval as jumping from 18 to 19 words). 3. Interval data (objective scientific scale): measured using universal, standardised physical units with equal intervals throughout the scale (e.g., reaction time in milliseconds, temperature in °C, heart rate in bpm, distance in centimetres).

Save

2

The 3-Criterion Decision Table for Choosing a Statistical Test

Always state all three reasons in context when justifying a statistical test: Criterion 1: Is it testing a difference between two conditions or a correlation (association) between two co-variables? Criterion 2: If a difference, is the design related (Repeated Measures or Matched Pairs) or unrelated (Independent Groups)? Criterion 3: Is the level of measurement nominal, ordinal, or interval? Memorise the grid: Nominal: Sign Test (Related Difference), Chi-Squared χ² (Unrelated Difference or Association between two categorical variables). Ordinal: Wilcoxon T (Related Difference), Mann-Whitney U (Unrelated Difference), Spearman's rho r_s (Correlation). Interval (Parametric tests, also require normal distribution and homogeneity of variance): Related t-test (Related Difference), Unrelated t-test (Unrelated Difference), Pearson's r (Correlation).

Save

3

Calculating the Sign Test Step-by-Step

The Sign Test is a non-parametric test used for a test of difference with a related design (repeated measures or matched pairs) when data is converted to nominal signs (+ or -). Step 1: For each participant, subtract their Condition B score from their Condition A score (A - B) and record only the sign (+ if A > B, - if A < B, or 0 if A = B). Step 2: Omit any ties (0s) and reduce the total sample size N by the number of ties. Step 3: Count the total number of plus signs (+) and the total number of minus signs (-). Step 4: The calculated value S is always the smaller of the two totals (the less frequent sign).

Save

4

Reading critical value tables: calculated value vs critical value

To check whether your calculated test statistic is statistically significant using a critical values table, you need three pieces of information: 1. Whether the hypothesis is one-tailed (directional) or two-tailed (non-directional); 2. The significance level (default in psychology is α = 0.05, or 5%); 3. The sample size N (or degrees of freedom df). Key rule on Direction of Comparison: For tests with an 'R' in their name (Spearman's rho, Pearson's r, Chi-SquarRed, Unrelated/Related t-test), the calculated value must be greater than or equal to (≥) the critical value to be significant. For tests without an 'R' in their name (Sign Test, Mann-Whitney U, Wilcoxon T), the calculated value must be less than or equal to (≤) the critical value to be significant.

Save

5

Probability, Significance Levels (p ≤ 0.05), and Writing Conclusions

In psychology, the standard significance level is p ≤ 0.05 (5%). This means that if we reject the null hypothesis (H₀), there is a 5% or less probability that the observed difference or correlation occurred purely by chance (random variation) if H₀ were actually true. Why 0.05? It strikes a practical balance between being too lenient and too strict. Only switch to the stricter p ≤ 0.01 (1%) level when the consequences of a false positive are dangerous or costly (e.g., testing a new medical drug with severe side effects, or a one-off study that cannot be replicated).

Save

6

Type I Errors (False Positives) vs Type II Errors (False Negatives)

Because statistical tests rely on probability rather than 100% certainty, two opposite errors can occur: 1. Type I error (False Positive / 'Optimistic Error'): Rejecting the null hypothesis (H₀) and claiming a significant effect when in reality H₀ is true and the result was just a fluke of chance. A Type I error is most likely when the significance level is set too LENIENTLY (e.g., p = 0.10 / 10%). 2. Type II error (False Negative / 'Pessimistic Error'): Accepting (failing to reject) the null hypothesis and missing a real effect when in reality the alternative hypothesis (H₁) is true. A Type II error is most likely when the significance level is set too strictly (e.g., p = 0.01 / 1%) or the sample size is too small.

Save

1. Q1

   A psychologist investigated whether a mindfulness app reduced stress levels in 10 employees. Each employee rated their stress out of 20 before using the app for a month and again after using the app for a month. Their scores (Before, After) were: P1 (16, 12), P2 (14, 15), P3 (18, 11), P4 (15, 15), P5 (13, 9), P6 (17, 13), P7 (12, 8), P8 (16, 16), P9 (19, 14), P10 (15, 10). Calculate the Sign Test statistic S and, given that the one-tailed 5% critical value for N = 8 is 1 and for N = 10 is 1, determine whether the mindfulness app significantly reduced stress. (6 marks)

   Check Mark Scheme

   Click “Check Mark Scheme” to compare your answer, 1. Calculate the sign of the difference (Before: After) for each participant: P1 (+), P2 (-), P3 (+), P4 (0/tie), P5 (+), P6 (+), P7 (+), P8 (0/tie), P9 (+), P10 (+) [1 mark]. 2. Count the signs: Number of '+' signs = 7; Number of '-' signs = 1; Number of '0' ties = 2 [1 mark]. 3. Identify the calculated value S as the frequency of the less frequent sign: S = 1 [1 mark]. 4. Exclude the 2 tied scores to find the effective sample size N: N = 10 - 2 = 8 [1 mark]. 5. Compare S with the one-tailed 5% critical value for N = 8 (which is 1): for the Sign Test, S must be less than or equal to (≤) the critical value; since S = 1 ≤ 1, the result is statistically significant at p ≤ 0.05 [1 mark]. 6. Reject the null hypothesis and conclude that employees had significantly lower stress ratings after using the mindfulness app for a month [1 mark].

2. Q2

   For each of the following three studies, name the most appropriate inferential statistical test and justify your choice using all three criteria: (a) A study correlating students' hours of sleep (measured to the nearest minute by a smartwatch) with their reaction time in milliseconds. (b) A study comparing the number of drivers who stop vs fail to stop at a zebra crossing when the pedestrian is wearing a suit vs casual clothes. (c) An independent groups study comparing memory scores out of 30 between participants given imagery training and a control group. (6 marks)

   Check Mark Scheme

   Click “Check Mark Scheme” to compare your answer, 1. (a) Pearson's r (Product Moment Correlation Coefficient): because the researcher is testing a correlation/association between two co-variables in a related design, and both sleep duration (minutes) and reaction time (milliseconds) are interval data measured on objective physical scales [2 marks]. 2. (b) Chi-Squared (χ²) test: because the researcher is testing a difference/association between two independent conditions (suit vs casual clothes), and the data is nominal (categorical frequency counts of drivers who 'stop' vs 'do not stop') [2 marks]. 3. (c) Mann-Whitney U test: because the researcher is testing a difference between two conditions using an independent groups (unrelated) design, and memory test scores out of 30 are treated as ordinal data (can be ranked, but intervals between test points are not guaranteed to be equal) [2 marks].

3. Q3

   A researcher conducts a correlation study with 20 participants using a two-tailed hypothesis and calculates a Spearman's rho value of r_s = +0.42. The critical values table for N = 20 shows: at p = 0.10 (two-tailed), critical value = 0.380; at p = 0.05 (two-tailed), critical value = 0.447; at p = 0.01 (two-tailed), critical value = 0.570. Determine whether the researcher's correlation is significant at the 5% level, and explain what type of error the researcher might be making if she decided to use the 10% (p = 0.10) significance level instead. (5 marks)

   Check Mark Scheme

   Click “Check Mark Scheme” to compare your answer, 1. Identify the correct critical value for a two-tailed test with N = 20 at the 5% (p = 0.05) significance level: critical value = 0.447 [1 mark]. 2. For Spearman's rho, the calculated value r_s must be greater than or equal to (≥) the critical value to be significant; since 0.42 < 0.447, the result is not statistically significant at the 5% level and the null hypothesis must be retained [2 marks]. 3. If the researcher switched to the more lenient 10% (p = 0.10) level (critical value = 0.380), 0.42 ≥ 0.380 so she would reject the null hypothesis [1 mark]. 4. Doing so risks a Type I error (a false positive): claiming there is a significant correlation in the population when in reality the null hypothesis is true and there is a 10% chance the correlation arose by random chance [1 mark].

4. Q4

   State the three conditions that must be met before a researcher is allowed to use a parametric statistical test (such as the Related t-test, Unrelated t-test, or Pearson's r). (3 marks)

   Check Mark Scheme

   Click “Check Mark Scheme” to compare your answer, 1. The data must be measured at the interval (or ratio) level using objective, standardised equal units [1 mark]. 2. The data must be drawn from a population that follows a normal distribution (bell-shaped curve with zero skew) [1 mark]. 3. There must be homogeneity OF variance - the spread (variance/standard deviation) of scores in the two conditions being compared must be approximately equal [1 mark].

Related study guides