Te Kete Ipurangi Navigation:

Te Kete Ipurangi
Communities
Schools

Te Kete Ipurangi user options:


You are here:

Effect size

A good way of presenting differences between groups or changes over time in test scores or other measures is by ‘effect sizes’, which allow us to compare things happening in different classes, schools or subjects regardless of how they are measured.

An effect size in educational measurement is the difference in assessment outcomes measured in standard deviation or standard error units. Essentially, calculating an effect size corrects for differences in the spread of data.

 

2007

Actual beginning of Year Results

Actual End of Year Results

Mean Diff

Confidence Interval for Effect Size

Effect Size

  Mean N SD Mean N SD   Lower Upper  
Year 9 Reading 505 81 56 602 68 67 97.0 1.21 1.95 1.59
Year 10 Reading 554 83 59 616 86 93 62.0 0.48 1.10 1.06
Year 9 Maths 507.7 82 65 613 67 106 105.3 0.87 1.58 1.23
Year 10 Maths 609.8 76 90 713 83 110 103.2 0.69 1.35 1.02

 

The mean is a measure of a central tendency. It is simply the average of all the values in a sample. To compute the mean, all the values are added and then divided by the number of values.

The standard deviation (SD) is a measure of the spread of the data. A smaller standard deviation would indicate the majority of results are close to that of the mean. A larger standard deviation would indicate that the results are more spread out, both below and above that of the mean.

The mean difference is the difference between the end of year results and the beginning of year results for each class.

The confidence interval for effect size is a measure of the significance of the effect size taking into account the spread of the data and also the number of observations. A confidence interval that doesn’t include zero indicates that there is a significant difference between students expected end of year score and students actual end of year score.

Effect size is a way of quantifying the difference between two groups. In this instance we are quantifying the difference between the end of year and beginning of year results.

An effect size of 1.0 indicates that a particular approach to teaching or technique advanced the learning of the students in the study by one standard deviation above the mean, typically associated with advancing children's achievement by one year, improving the rate of learning by 50%, or a correlation between some variable (for example, amount of homework) and achievement of approximately .50. When implementing a new programme, an effect size of 1.0 would mean that approximately 95% of average students receiving that treatment would exceed 84% of students not receiving that treatment (Hattie, 1999).

According to Hattie (1999) normative comparison points of effect sizes are:

  • 0.1 Student maturation
  • 0.24 Any teacher in front of class
  • 0.4 Innovations in schooling

Anything above 0.4 would imply that the innovation is working better than expected.

The Professional Learning and Development BES suggests 0.2 – 0.4 is a small but educationally significant impact, 0.4 – 0.6 is a medium educationally significant impact and greater than 0.6 is large.

Further reading