IB Psychology HLTopic 5 — Research MethodsPaper 3 & IACore skill~10 min read
Questionnaires and Survey Design
A questionnaire looks like the easiest method in psychology. Write some questions, hand them out, count the answers. The catch is that people are not very good at reporting themselves honestly — and a badly worded question can create a result that was never really there.
📚 What you need to know
A survey is the whole data-gathering exercise. A questionnaire is the set of questions used to do it.
Questionnaires are a form of self-report: participants describe their own thoughts, feelings and behaviour.
Closed questions limit the answer and produce quantitative data.
Open questions allow free responses and produce qualitative data.
The big weakness is social desirability bias: people under-report the bad and over-report the good.
Standardised questions make a questionnaire easy to replicate, which is good for reliability.
Reliability is checked with test-retest (over time) and split-half (within the questionnaire).
Closed and open questions do different jobs
The choice is not about which is better. It is about what kind of answer your research question needs. If you want to know how many, use closed questions. If you want to know why, use open ones. Most decent questionnaires use both.
This is the reason questionnaires are so often criticised for being shallow: researchers use closed questions because they are quick to analyse, then have nothing to say about why.
A common question asks you to write one closed and one open question on the same topic. Do exactly that — same topic, different formats. Writing two closed questions in different words gets you nothing.
The honesty problem
Self-report only works if people tell you the truth. Often they do not, and usually they are not even lying on purpose. They are shaping the answer into the person they would like to be. Psychologists call this social desirability bias, and it hits questions about health, effort, prejudice, money and anything with a socially “right” answer.
Reliability and validity come apart here. Ask the same person twice and you get the same flattering answer both times — consistent, and still wrong.
How to write questions that do not break
Four things wreck a questionnaire more than anything else. Learn them as a checklist and you will spot them instantly in an exam question.
Problem
What it looks like
Why it damages the data
Leading question
Do you agree that homework is pointless?
Tells the participant which answer is expected, so it measures obedience, not opinion
Double-barrelled
Do you find school stressful and tiring?
One answer, two questions. You cannot tell which half they agreed with
Jargon or vague words
Do you experience frequent dysregulation?
Different people read it differently, so responses are not comparable
Question order bias
Emotive questions placed first
Early questions frame the mood and shift every later answer
The order fix is simple: start neutral and general, then move to the specific and sensitive. Warm people up before you ask them anything they might defend themselves against.
Checking a questionnaire is reliable
Reliability means consistency, and there are two ways to test for it. Test-retest gives the same questionnaire to the same people after a gap — six months is typical — and checks whether the scores match. That is external reliability. Split-half takes one set of responses, splits the questionnaire into two halves, and checks whether both halves give a similar score. That is internal reliability: the items are all measuring the same thing.
🧩 Building a questionnaire for the IA
Start from the construct. Write down exactly what you are trying to measure before you write a single question.
Mostly closed items, so the data can actually be analysed with a statistical test.
Add two or three open items at the end to explain any pattern you find.
Read each item out loud. If it contains “and”, “obviously”, or a word you had to look up, rewrite it.
Pilot it on a few people. Ask what they thought each question meant. Their answers will surprise you.
Guarantee anonymity in writing. It is the cheapest available defence against social desirability bias.
Worked examples
WORKED EXAMPLE
Identify the flaws and rewrite the item
A researcher studying student wellbeing includes this item: “Don’t you agree that social media is harmful and makes teenagers anxious?” Identify two problems and rewrite the question.
Step 1: First problemLeading question — “Don’t you agree” signals the expected answer, so agreement is inflated.
Step 2: Second problemDouble-barrelled — it asks about harm and about anxiety at once, so a single answer cannot be interpreted.
Step 3: Name the cost
Both reduce the validity of the responses: the item no longer measures the participant’s real view.
Step 4: Rewrite as one neutral idea
“How often does using social media make you feel anxious?” with options: never, rarely, sometimes, often, always.
Leading + double-barrelled; split into one neutral scaled itema rewrite must fix both faults, or you only get half the marks
WORKED EXAMPLE
Which reliability check, and why?
A team develops a 20-item stress scale. They want to be sure the items all measure stress and are not drifting onto other topics. They also want to know that a person’s score would not swing wildly from one month to the next. Explain which reliability check answers each concern.
Step 1: Concern one is about the items themselves
Use the split-half method: split the 20 items into two sets of 10 and compare the two scores.
Step 2: Name what that shows
A close match means high internal reliability — the scale is consistent within itself.
Step 3: Concern two is about stability over time
Use test-retest: give the same scale to the same people after a set gap.
Step 4: Name what that shows
Similar scores mean high external reliability.
Split-half for internal, test-retest for externalinternal = inside the test, external = across time. Same words, different job
💡 Exam tip
Name the bias, do not just describe it. “Social desirability bias” is worth more than “people might lie”.
Always pair a criticism with the validity or reliability it damages. That link is the evaluation mark.
When asked for a strength of questionnaires, “large samples quickly and cheaply” plus “standardised, so replicable” covers most mark schemes.
Anonymity is the standard suggested improvement. Say why it works: it removes the reason to impress the researcher.
Closed questions give you numbers, so they let you run a statistical test. Mention that if the question is about analysis.
Do not confuse a survey with a questionnaire in your writing. The questionnaire is the tool inside the survey.
⚠ Common mix-up
Thinking open questions are “better”. They give richer data but are slow to analyse and much harder to compare across participants.
Calling a questionnaire a method that shows cause and effect. No IV is manipulated, so it cannot.
Mixing up internal and external reliability. Split-half is internal, test-retest is external.
Confusing social desirability with demand characteristics. Social desirability is about looking good. Demand characteristics are about guessing the aim and playing along.
Assuming a big sample fixes bias. A larger sample makes a biased result more precise, not more accurate.
Writing “quantitative” when you mean closed. Closed is the question type; quantitative is the data it produces.
Up next: Interviews and What They Reveal — the same self-report problem, but this time with a real person sitting opposite you.
Want this explained one-to-one?
Book a free session with an experienced IB Psychology tutor and get your trickiest topics made simple.
