Experimental designs: repeated measures, independent groups, matched pairs. Control: random allocation and counterbalancing, randomisation.

EXPERIMENTAL DESIGN DEFINITION:

Experimental design refers to how participants are allocated to the different groups in an experiment. Types of design include repeated measures, independent groups, and matched pairs designs.

The classic experimental design definition is, “The methods used to collect data in experimental studies.”

 The way you classify research subjects, based on conditions or groups, determines the type of design. There are two primary types of experimental design: True experimental research design. Quasi-experimental research design

 True experimental research design: True experimental research relies on statistical analysis to prove or disprove a hypothesis, making it the most accurate form of research. Of the types of experimental design, only true design can establish a cause-effect relationship within a group. In a true experiment, three factors need to be satisfied:

• There is a Control Group, which won’t be subject to changes, and an Experimental Group, which will experience the changed variables.

• A variable that can be manipulated by the researcher

• Random distribution

• This experimental research method commonly occurs in the physical sciences.

3. Quasi-experimental research design: The word “Quasi” indicates similarity. A quasi-experimental design is similar to an experimental, but it is not the same. The difference between the two is the participants of a group are not randomly assigned.

INDEPENDENT GROUP DESIGN

Independent Group design

You guessed it. If we used the same people in each group last time, this time we use different people in each group.  Clearly this overcomes practice and boredom effects because they only do it the once!

Each participant is randomly allocated to one group or the other, so in our coffee experiment:

One group, comprising one set of participants do the test with coffee

The other group, comprising a different set of participants, does the test without coffee.

Sorted, no problems with practice or repeat effects or with boredom or tiredness effects.

However

Can we be certain that the likely faster reactions of the first group are down to the coffee?

It could be that the participants that we’ve randomly assigned to that condition have naturally faster reactions.  They may be younger, or some of them may engage in activities that require fast reactions. 

In other experiments, sex, personality, age, IQ etc. could all be an issue because the participants are going to differ on all of these.

There are some occasions when independent measures design has to be used:

Sex differences

Age differences

By definition the two conditions are different.  You couldn’t have someone in the male condition and the female condition, or in the under 30 condition and the over 30 condition!

Advantages

1.       No order or practice effects

2.       Can use the same stimulus material (such as word lists in memory) for each group

Disadvantages

1.       Participants are not matched in terms of IQ, personality, age etc.

2.       You will need twice as many participants. 

3.       The researcher may be biased in assigning participants to conditions. Therefore he/she should randomly allocate participants by using random number tables.

For example, computer generated random number tables and assigning random numbers to participants.

Clearly these are the same as the advantages and disadvantages of repeated measures but in reverse.

REPEATED MEASURES DESIGN DEFINITION:

Repeated Measures design is an experimental design where the same participants take part in each condition of the independent variable. This means that each condition of the experiment includes the same group of participants. Repeated Measures design is also known as within groups, or within-subjects design

EXAMPLE: REPEATED MEASURES DESIGN APPLIED TO RESEARCH

INTRODUCTION

• A SUMMARY OF AVAILABLE RESEARCH: Many people have heard of the “Mozart Effect” or an idea similar to it. This hypothesis questions whether a person who listens to the compositions of Mozart before a test will score higher on the test than a person who does not. It’s theorised that music increases beta-waves and beta-waves increase attention and attention increases intelligence. In addition to this, hearing is lateralised, that is there are different sound specialisations on each half of the brain. The left hemisphere has an advantage for decoding speech and the right hemisphere has an advantage for decoding music and other non-human noise.

• JUSTIFICATION: The researcher hypothesises that because speech and music are processed independently in the brain, that music should not affect performance (as long as the music does not have lyrics). Moreover, because music increases beta-waves, cognitive tasks should be easier.

• AIMS: A researcher develops a theory on the basis of this finding and wonders if listening to music will affect tasks that require thinking or analysing, e.g., listening to the radio whilst writing an essay or solving a maths problem. Because there was previous research, a directional (1-tailed) hypothesis was chosen.

The researcher created the following null and alternative/experimental hypotheses:

• NULL HYPOTHESIS: There will be no difference between participants in the “music condition” and participants in the “no music” condition and their score on an IQ test.

• ALTERNATIVE HYPOTHESIS: Participants in the “music condition” will score higher on an IQ test than participants in the “no music condition”

METHOD

DESIGN: A laboratory experiment with a Repeated Measures Design. The independent variable is music operationalised as Mozart's Sonata No. 11 and the control condition is no music. The dependent variable is the scores on the Stanford Binet IQ test.

The conditions are either:

•  Taking an IQ test whilst listening to music

•  Taking an IQ test whilst listening to no music

PROCEDURES: The researcher conducts a pilot study with ten volunteer participants from a local SEN school.

RESULTS

DESCRIPTIVE STATISTICS

Participants take condition A first. Notice how high the results are in condition A.

REPEATED MEASURES DESIGNS are affected by ‘order effects’, e.g., the order the participants take the experimental conditions (IVS) and controls in. In plain speak, “should the participants take the music condition first” or “should the participants take the no music condition first”. Will there be a difference if one condition is chosen over the over?

Actually, it doesn’t matter which condition you choose to go to first, both are wrong.

The first reason is below.

BOREDOM EFFECTS: Look at the data in table 1. Condition A has done much better than condition B. This could be interpreted as the experiment has failed, e.g., accepting the null hypothesis. But this is unlikely because the researcher has not controlled for the order participants take the conditions in, e.g., the ORDER EFFECTS.

Participants could be eager to please or excited to do well in the first condition. The first condition is a new experience and they might give it their best shot.

But because participants gave it their “all” in condition one, sometimes they can’t be bothered to do the same thing again in condition two because they have become bored. The results then could show condition one doing much better and the researcher thinks it’s to do with the IV but it is to do with order effects.

The second reason is below.

EXPERT EFFECTS: Alternatively, participants might be unsure of what they have to do in the first condition, they have never done an IQ before and as result, they do badly in condition A. Then in the second condition, they become better perhaps even experts because they now know what to do. The results in Condition B would then show condition two doing better as they do in Table 2 below. The researcher thinks it’s to do with the IV but it is to do with order effects

Participants take condition A first. Notice how high the results are in condition B SOLUTION: COUNTERBALANCE!

Researchers counterbalance to reduce order effects. In short, it means to randomly allocate participants in a repeated design group into two different groups; e.g., a group one and a group two.

·      Then make group one, do condition A first and make group two do condition B first.

·      Then make group one, do condition B second and group two, do condition A second. 

The groups are now counterbalanced, for example, if participants get bored in condition two then some will be bored whilst music is playing and others while music is not playing. If Participants become experts in condition two then again some will be experts whilst music is playing and others, while music is not playing.

The counterbalancing of groups eliminates the bias, e.g., boredom is experienced in both conditions so can’t skew one set of data.

REPEATED MEASURES COUNTERBALANCED

Experimental research design

Here we decide how we are going to sort or group our participants.  Do we use the same people in all conditions or groups, or do we choose different people for different conditions or groups?  In some cases, as we’ll see the decision is made for us.  In others the solution isn’t so obvious and there may be pros and cons for each.

Repeated Measures Design

Here we use the same participants in each group or condition. 

For example, returning to the earlier experiment on coffee and reaction times.

In a repeated measures design we could give our group of participants the test on day one with no coffee and record their reaction times.

The next day we could repeat the procedure, with the same group of people, but this time give them coffee before the experiment began.

Advantages

The two groups have the same age, sex, personality, ideas, past experiences, IQ, reaction times (crucially for this one) etc.  They are perfectly matched.  They are the same people!

Less people are needed

Disadvantage

Order effects: Assuming, as we expect the group do better on the second day, can we be sure that this increase in performance is due to the coffee?  It could be that they’ve had the chance to practice the task the day before!  It’s not surprising they’re better the second time around.  This is called order or practice effect. 

Boredom: Of course, on some tasks it could work the other way, and a task done the second time shows deterioration because they’re fed up with doing it.

Extra materials: For example if you use the same participants for two memory experiments you will need two lists of words etc. for them to recall.  This introduces other variables.  Perhaps the second list is easier than the first.

 Demand characteristics. They have already done one condition they will probably guess what the experiment is about.

MATCHED PAIRS DESIGNS

BETWEEN GROUP QUASI DESIGN

Quasi Experiments: Definition

Quasi experiments are when participants cannot be randomly allocated to conditions. This can happen in field; lab or natural experiments (remember not all quasi experiments are natural). It can happen because it is not possible to randomly allocate (natural/field or the researcher forgot (bad research). Not randomly allocating participants to conditions is a bias. You may have uneven groups or choose participants with characteristics that support your hypothesis.

Some people consider that studies of gender differences would be quasi experiments. An example of a study of gender differences would be comparing whether boys and girls have higher IQS. It might be claimed that gender is the IV and IQ is the DV. However the variable ‘gender’ has not been manipulated (or changed) it occurs naturally. As we already know, IV must be manipulated in order to be considered true experiments. In reality we should call gender just a variable and IQ just a variable. The same is true of studies looking at other differences between groups: e.g., extroverts and introverts, blondes and brunettes, smokers and non-smokers etc. – these conditions are not applied to the person they are an existing part of them. Such studies are also called quasi experiments. We cannot draw causal conclusions. We cannot say for example that being blonde caused an individual to have a higher IQ that a redhead. We can only conclude that hair colour is related to IQ. There are no obvious advantages and disadvantages as quasi can be so different

Quasi Experiments: Definition

Quasi experiments are when participants cannot be randomly allocated to conditions. This can happen in field; lab or natural experiments (remember not all quasi experiments are natural). It can happen because it is not possible to randomly allocate (natural/field or the researcher forgot (bad research). Not randomly allocating participants to conditions is a bias. You may have uneven groups or choose participants with characteristics that support your hypothesis.

Some people consider that studies of gender differences would be quasi experiments. An example of a study of gender differences would be comparing whether boys and girls have higher IQS. It might be claimed that gender is the IV and IQ is the DV. However the variable ‘gender’ has not been manipulated (or changed) it occurs naturally. As we already know, IV must be manipulated in order to be considered true experiments. In reality we should call gender just a variable and IQ just a variable. The same is true of studies looking at other differences between groups: e.g., extroverts and introverts, blondes and brunettes, smokers and non-smokers etc. – these conditions are not applied to the person they are an existing part of them. Such studies are also called quasi experiments. We cannot draw causal conclusions. We cannot say for example that being blonde caused an individual to have a higher IQ that a redhead. We can only conclude that hair colour is related to IQ. There are no obvious advantages and disadvantages as quasi can be so different.