You are here:

• Home »
• Using evidence for learning »
• Gathering evidence »
• The evidence-driven decision making cycle »
• Analyse

Analysing data and other evidence is an important step in the process. But it’s not the most crucial one. If you don’t ask very good questions, no amount of data analysis will give you useful answers.

All schools have access to tools that will generate, for example, gender and ethnicity comparisons, intra subject analyses, comparisons with national results. (See links below.)

Most schools need some staff members who are responsible for leading data analysis. But classroom teachers can have access to electronic tools to process data into graphs and tables.

Of course, all teachers do some data analysis. This set of results for year 12 students in one subject can be ‘cut’ in many ways even without access to statistical tools. Classroom teachers could:

• divide the class into three groups on the basis of overall achievement
• identify students who are doing so well at level 2 that they could be working at a higher level
• find trends for males and females, those who are absent often, or have many detentions
• compare this group’s external assessment success rate with the national cohort.

*indicates externally assessed

A guide to data analysis can be found on Te Kete Ipurangi.

Whoever does the analysis, significance and confidence are key issues.

Even if you don’t fully understand how the analysis was carried out, you should still think critically about the results. It’s important that every teacher develops the ability to make sense of the results of analysis.

How significant are any apparent trends?

How much confidence can we have in the information?

Ask the person who did the analysis what has been done, what they think the results show, what limitations they would place on the results.

The table below is an example of an application of an analysis tool that any teacher could use. It shows how reading levels have changed within the school year.

(In asTTle test scores, level 5B is higher than level 4B. More information on asTTle testing can be found on Te Kete Ipurangi.)

Any teacher could interpret this information in the table below. But care is needed about conclusions reached.

Reading levels – terms 1 and 4

The table shows that reading levels overall were higher in term 4 than in term 1. Scores improved for most students. Over 20% of students moved into Level 5.

All of this sounds positive. But if you work out the median score, you’ll see that it is unchanged. The ‘middle’ student in this class is still on 4A.

If you cannot view or read this graph, select this link to open up a text version.

Is this information? Can we act on it?

This is useful summative information for reporting against targets and as general feedback to teachers. But it’s not information teachers could act on to improve student achievement.

In fact, is it actually 'information'? In a sense, this is still data – a graphical representation of two sets of related data.

Think about what counts as information. Here is the definition used in this resource:

Information is knowledge gained from analysing data and making meaning from evidence.

There’s a fuller discussion in Terminology.

Here is a test of whether you really have information: Does whatever you have give you knowledge or understanding to inform your decision-making?

How confident you will be about this knowledge depends on a number of factors – where the data came from, how reliable it was, how rigorous your analysis was.

Information can be summative or formative:

• Summative information is useful for reporting against targets and as general feedback to teachers.
• Formative information is information we can act on – it informs decision-making that can improve learning.

To get formative information, information that will inform decision-making, from the asTTle graph shown earlier, you would need to ask (and find answers for) more questions:

• How have the scores of individual students changed?
• Did the more able students make significant progress, but not the lower quartile?
• How many remained on the same level?
• How much of this shift in scores is attributable to students’ predictable progress? Is there any data that will enable us to compare our students with a national cohort?
• How does this shift compare with previous year 9 cohorts?
• How much have our teaching approaches contributed to this result?

You can’t answer these questions from the graph. You would need to disaggregate the data to get information you could act on.

(In fact, this graph has been created by aggregating separate asTTle results – so you’d probably go back to the original data and aggregate it in different ways.)

How significant is the information you get from analysing data? Is it a conclusion, a trend, a possibility?

It’s worth thinking about how you express information, especially in relation to the question that triggered the analysis.

The verbs you choose reflect the confidence you have in the information. You can even see a hierarchy. Information can: establish, indicate, confirm, reinforce, back up, stress, highlight, state, imply, suggest, hint at, cast doubt on, contradict, refute …

When you interrogate information you will ask even more questions.

• Does this confirm that …?
• What does this suggest?
• What are the implications of …?
• How confident are we about this conclusion?

The answers will often come from your professional judgment.

Back

Return to top