Student achievement in the Observation Survey is a measurement of reading text level (90% or more accuracy) and the stanine scores relating to the five ‘test’ elements of the assessment (letter identification, concepts about print, writing vocabulary, word reading, and hearing and recording).
The graph below shows the alignment between reading text level (90% or more accuracy) and the National Standard for Reading after one year.
Likelihoods of Scripts being Judged at Different Reporting Bands Depending on Text Level
The horizontal axis shows the test score and the vertical axis the probability, based on the judgments made by the team, that a score or colour band will be within a certain standard.
To read the graph, consider a vertical line above a text level of, for example, 11 (illustrated). This is the upper blue text in the colour wheel. Looking at the areas with which the line intersects, we can conclude the following.
- Almost no part of that line is in the blue area, reflecting that a script with a text level of 11 had almost no likelihood of being judged as above the ‘after one year’ national standard in reading.
- A medium sized part of that line is in the green area, reflecting that a script with a text level of 11 had a moderate likelihood of being judged as at the ‘after one year’ national standard in reading.
- A large part of that line is in the orange area, reflecting that a script with a text level of 11 had a large likelihood of being judged as below the ‘after one year’ national standard in reading.
- Only a very small part of that line is in the red area, reflecting that a script with a text level of 11 had a very small likelihood of being judged as well below the ‘after one year’ national standard in reading.
However, using the Observation Survey to help make an overall teacher judgment is made more complex by the five different ‘test’ elements, reported in stanines. Because the five stanines relate to different aspects of reading achievement, an overall average is not very informative when making a judgment against the standard, so the data here is presented as examples of actual results from scripts in each reporting band, in ‘star plots’ as shown below.
There are some interesting patterns to be observed, which are picked out alongside each plot. In reading the plots, the further a line is from the centre of the ‘star plot’, the higher the stanine. It should be noted that some assessments have maximum scores which correspond to more than one stanine, and in this case the lowest value has been taken in these plots. For Letter Identification and Hearing & Recording the upper limit is taken to be stanine 7; for Word Reading it is 6.
Scripts judged well below standard
The scripts judged well below standard showed consistently low stanines on Word Reading and Writing Vocabulary despite some higher stanines on the other aspects.
Scripts judged below standard
The scripts judged below standard showed most aspects around stanine 4 or 5.
Scripts judged at standard
The scripts judged at standard reflected high or maximum stanines on all aspects, except for Writing Vocabulary.
Scripts judged above standard
The scripts judged above standard reflected maximum stanines on most aspects and high stanines on Writing Vocabulary.
The results from the ‘star plots’ indicate that Writing Vocabulary stanine scores, in conjunction with those of Concepts about Print, were the main factors in scores being judged within each reporting band.
In summary, to use the Observation Survey to assist in making overall teacher judgments, teachers will need to take into account the reading text level as assessed from the Running Record and the stanine scores from the five ‘test’ aspects, but in particular the results from the Writing Vocabulary aspects of the assessment in conjunction with the Concepts about Print.
The Observation Survey and the National Standards
(PDF 1 MB)