Research Methods: Experimental Designs, Variables, Sampling, Reliability & Validity

1

Variables (IV, DV, Extraneous, Confounding) & Operationalisation

In an experiment, the researcher manipulates the independent variable (IV) and measures its effect on the dependent variable (DV). OPERATIONALISATION means defining both the IV and DV in concrete, measurable, and observable terms (e.g., instead of 'better memory', write 'the number of words correctly recalled out of 20 from List A in 2 minutes'). An extraneous variable (EV) is any nuisance variable other than the IV that could affect the DV if not controlled (including participant variables like age/IQ and situational variables like room temperature, noise, or time of day). If an EV varies systematically with the IV so you cannot tell whether the IV or the EV caused the change in the DV, it becomes a confounding variable (destroying internal validity).

Save

2

Directional vs Non-Directional Hypotheses

The Null Hypothesis (H₀) always predicts NO significant difference or correlation ('There will be no significant difference in mean number of words recalled out of 20 between participants who study in silence and those who study with music; any difference is due to chance'). For the Alternative Hypothesis (H₁): 1. Use a directional (one-tailed) hypothesis only when previous research theory or findings clearly point in a specific direction ('Participants who study in silence will recall significantly more words out of 20 than.'). 2. Use a non-directional (two-tailed) hypothesis when there is no previous research or when previous findings are contradictory ('There will be a significant difference in the number of words out of 20 recalled between.').

Save

3

Types of Experiments: Lab, Field, Natural, and Quasi

1. Laboratory experiment: Conducted in a highly controlled artificial environment where the researcher directly manipulates the IV (Strength: high internal validity and easy replication; Limitation: low mundane realism/ecological validity and high risk of demand characteristics). 2. Field experiment: Conducted in a natural, everyday setting where the researcher still directly manipulates the IV (Strength: higher ecological validity and less awareness of being studied; Limitation: less control over extraneous situational variables and ethical issues of lack of informed consent). 3. Natural experiment: The IV is an event or condition that varies naturally without the researcher manipulating it (e.g., Rutter's Romanian orphan study or Charlton's study of introducing TV to St Helena). 4. Quasi-experiment: The IV is a pre-existing internal characteristic of the participants (such as biological sex, age group, or having a phobia/schizophrenia diagnosis) so participants cannot be randomly allocated to conditions.

Save

4

Experimental Designs: Independent Groups, Repeated Measures & Matched Pairs

1. Independent groups: Different participants are allocated to each condition of the IV. Strength: no order effects (practice or fatigue) and less chance of guessing the aim; Limitation: participant variables between groups may confound results (Fix: Random ALLOCATION using a random number generator or lottery, though twice as many participants are needed). 2. Repeated measures: Every participant completes all conditions of the IV. Strength: zero participant variables between conditions and requires half the participants; Limitation: Order effects (boredom/fatigue or practice improvement on the second task) and participants easily guess the aim (Fix: Counterbalancing - ABBA - where half the sample does Condition A then B, and the other half does B then A; Note: counterbalancing does not eliminate order effects, it balances them equally across both conditions). 3. Matched pairs: Participants are pre-tested on key variables (e.g., IQ, age, memory score), paired up with a near-identical twin/match, and one member of each pair is randomly assigned to Condition A and the other to Condition B (avoids order effects and reduces participant variables, but matching is time-consuming and never 100% perfect).

Save

5

Sampling Methods (Random, Systematic, Stratified, Opportunity, Volunteer)

A sample is drawn from a target population: 1. Random: Every member of the target population has an equal chance of selection (assign everyone a number and use a random number generator/lottery; unbiased, but time-consuming and participants may refuse). 2. Systematic: Pick every nth name from a sampling frame list (e.g., every 10th name). 3. Stratified (most representative): Divide the target population into subgroups (strata, e.g., age or gender), calculate the exact percentage of each stratum in the population, and randomly select from each stratum in those exact proportions. 4. OPPORTUNITY: Select whoever is conveniently available at the time (fast and cheap, but biased by time/location). 5. Volunteer (Self-selecting): Participants respond to an advert (easy, but suffers from volunteer bias as volunteers are more eager to please/inquisitive).

Save

6

Reliability and Validity (Assessment and Improvement)

Reliability is the consistency of a measure: 1. Test-Retest Reliability (administer the same questionnaire/test to the same participants on two separate occasions and correlate the two sets of scores, must reach r ≥ +0.80; improve by replacing ambiguous open questions with clear closed fixed-choice items). 2. Inter-Observer / Inter-Rater Reliability (two independent observers use the same behavioural checklist simultaneously during a pilot study and correlate their tallies, must reach r ≥ +0.80; improve by operationalising behavioural categories so they do not overlap). Validity is whether a study measures what it claims to measure and represents real life: Internal Validity (threatened by confounding variables, demand characteristics, and investigator effects, fix with single-blind/double-blind procedures and standardised instructions), Ecological Validity (generalisability to real-world settings), Temporal Validity (across historical eras), and Concurrent Validity (correlating a new test with an established gold-standard test for ≥ +0.80).

Save

1. Q1

   A psychologist wants to investigate whether classical music improves revision compared to silence. She tests 20 sixth-form students on Monday afternoon in a quiet classroom where they study List A (15 two-syllable nouns) for 5 minutes in silence and then write down as many words as they can remember. On Tuesday morning, the same 20 students study List B (15 two-syllable nouns) for 5 minutes while Mozart plays and then write down as many words as they can remember. (a) Identify the experimental design used and explain one limitation of this design in the context of this study. (b) Explain how the psychologist should use counterbalancing to address this limitation. (c) Identify one extraneous situational variable in the psychologist's procedure and explain how it could have confounded the results. (7 marks)

   Check Mark Scheme

   Click “Check Mark Scheme” to compare your answer, 1. (a) The experimental design is a Repeated Measures design (because the same 20 students participate in both the silence and classical music conditions) [1 mark]. 2. Limitation in context: Order effects - by Tuesday morning (the music condition), students may perform better on List B simply because they have practised the memory task technique on Monday, OR perform worse due to boredom/interference from Monday's word list [2 marks]. 3. (b) Divide the 20 students randomly into two equal groups of 10: Group 1 (10 students) completes the Silence condition (List A) first followed by the Mozart condition (List B), while Group 2 (10 students) completes the Mozart condition (List A) first followed by the Silence condition (List B) (also counterbalancing the word lists so List A and List B appear equally in both conditions) [2 marks]. 4. (c) Time of day (Monday afternoon vs Tuesday morning): students may be more alert and refreshed on Tuesday morning than on Monday afternoon [1 mark]. 5. Because time of day varied systematically with the IV (all silence trials in the afternoon, all music trials in the morning), any higher score in the music condition could be caused by morning alertness rather than Mozart [1 mark].

2. Q2

   A target population of 800 hospital staff consists of 480 nurses, 200 doctors, and 120 administrative staff. A researcher wants to select a stratified sample of 40 staff to interview about workplace stress. Calculate how many nurses, doctors, and administrative staff should be selected, and describe how the researcher should select the specific individuals from each group. (4 marks)

   Check Mark Scheme

   Click “Check Mark Scheme” to compare your answer, 1. Calculate the proportion of each stratum in the sample of 40 using (Stratum size ÷ 800) × 40 [1 mark]. 2. Nurses: (480 ÷ 800) × 40 = 24 nurses; Doctors: (200 ÷ 800) × 40 = 10 doctors; Administrative staff: (120 ÷ 800) × 40 = 6 administrative staff (Check: 24 + 10 + 6 = 40) [2 marks]. 3. For each stratum separately, assign every member a unique number (e.g., 1 to 480 for nurses) and use a computer random number generator (or names in a hat) to select the required number (24 nurses, 10 doctors, 6 admin staff) at random [1 mark].

3. Q3

   A researcher develops a new questionnaire to measure 'Examination Anxiety' in A-Level students. Explain how the researcher could assess: (a) the test-retest reliability of the questionnaire, and (b) the concurrent validity of the questionnaire. Include the statistical threshold required in both cases. (6 marks)

   Check Mark Scheme

   Click “Check Mark Scheme” to compare your answer, 1. (a) Test-Retest Reliability: Administer the new Examination Anxiety questionnaire to the same sample of A-Level students on two separate occasions (leaving a gap of 2 to 4 weeks so they do not simply remember their previous answers, but before their actual anxiety changes) [1 mark]. 2. Calculate a correlation coefficient (Spearman's rho or Pearson's r) between each student's Test 1 score and Test 2 score [1 mark]. 3. If the positive correlation is +0.80 or higher (r ≥ +0.80), the questionnaire has high test-retest reliability [1 mark]. 4. (b) Concurrent Validity: Administer both the new Examination Anxiety questionnaire and an already-established, validated anxiety scale (such as Spielberger's State-Trait Anxiety Inventory) to the same sample of students at the same time [1 mark]. 5. Correlate the two sets of anxiety scores for each student [1 mark]. 6. If the correlation between the new questionnaire and the established test is +0.80 or higher (r ≥ +0.80), the new questionnaire has high concurrent validity [1 mark].

4. Q4

   Explain the difference between a structured observation and an unstructured observation, and explain why a researcher studying playground aggression in 5-year-olds might choose a covert, non-participant, structured observation. (5 marks)

   Check Mark Scheme

   Click “Check Mark Scheme” to compare your answer, 1. In an unstructured observation, the observer records a continuous narrative of all relevant behaviours seen without pre-set categories, whereas in a structured observation, the observer uses a pre-determined behavioural checklist and sampling frame (event or time sampling) to tally specific operationalised behaviours [2 marks]. 2. Covert (hidden, e.g., watching from inside a classroom window): ensures the 5-year-old children do not know they are being watched, eliminating participant reactivity / demand characteristics so their playground behaviour remains natural [1 mark]. 3. Non-participant: an adult researcher cannot realistically join in as a 5-year-old child on the playground without disrupting the natural play dynamic [1 mark]. 4. Structured: using a pre-agreed checklist of operationalised aggressive acts (e.g., 'hitting', 'snatching a toy', 'shoving') produces quantitative data that is easy to compare and ensures high inter-observer reliability [1 mark].

Related study guides