Alcoa Champions of Maths – the Finale!

Aloca Champions of Maths (ACoM) has consistently demonstrated the power to significantly improve teacher and student outcomes, in beliefs, behaviour, and performance. In this final blog, I’d like to share with you the outstanding results we had in 2021 with our evaluation of the program. 

Positive change in teacher confidence and beliefs 

As the content of the program focused specifically on problem solving and reasoning in maths classes, teachers were asked to rate their agreement with two statements on a Likert scale from Strongly Disagree to Strongly Agree, on a scale of 0 to 4.  Changes in their response from beginning to end of year showed: 

I have a high level of confidence in teaching problem solving in mathematics. 34% average increase. 

I have a high level of confidence in assessing reasoning. 41% average increase. 

We have seen similarly strong results across all three years of the implementation of ACoM. 

A strong shift in the teachers’ confidence in these two areas supports the idea that the program has met its intended aims of increasing capacity in teachers through problem solving and reasoning. Further evidence of this was through the large collection of lesson plans and blog posts created by teachers over the year. 

Teachers were also measured, through a survey and scored on a rank of 0-14, on how connectionist their orientation (belief set) was. A leading piece of research from the UK in 1997 showed that teachers who held a strongly connectionist orientation achieved higher student gains than teachers who held either discovery or transmission orientations. A teacher with a connectionist orientation believes that numeracy is best developed through discussion, connecting different mathematical ideas explicitly, and a focus on efficiency in problem-solving; these are all beliefs reinforced by the methods employed in ACoM. A discovery orientated teacher focusses on allowing students to build their own understandings, and a transmission-oriented teacher focusses on explicitly teaching students to follow specific procedures. 

Over the year, we observed an average shift of 14% toward connectionist orientation. The scale caps at 14, and teachers had an average score of 11.5 at the end of the program. This indicates a very strongly connectionist group of teachers. This is an extremely satisfying result and indicates a significant, desirable shift in teachers’ beliefs that is known to be associated with higher student outcomes. 

Positive change in teacher behaviour: 

When asked at the close of the program how their relationship with mathematics and their teaching had changed, teachers shared inspiring reflections. This is a sample of comments made: 

My teaching has changed drastically. 

I have developed an even stronger passion for mathematics. 

I have become more confident in tackling mathematics concepts. 

I am less intimidated by maths as well as teaching maths. 

I have become more aware of how capable all students are of tackling maths problems. 

Now I understand that they need to struggle to learn. They need to discuss and collaborate with their peers so they can not only understand their own strategies but others and compare which is more efficient. 

I can anticipate student responses more accurately which allows me to ask more productive, effective questions. 

My planning is more fluid or adaptable and allows for genuine extension and looking closely at misconceptions. 

Now I give them learning experiences to expand their thinking and reasoning skills. 

Positive change in student learning: 

Teachers were also asked to give their impression of student’s engagement and understanding of problem-solving and reasoning during the pre and post surveys. Results from this survey showed a positive increase of 16% in student engagement and 19% in student’s understanding of problem-solving and reasoning. 

All teachers participating in the program used the 4th edition PAT-Maths tests. These are a standardised assessment that tests general knowledge across the whole curriculum for each year level. The tests are normalised across year levels, so we can identify expected scores and growth from beginning to end of year. Based on these norms, and an unpublished model developed by Tierney Kennedy of Back-to-Front Maths, we were able to obtain an estimate of gains during the year compared to the national average expected gains. These are the key results for six classes (two of the eight did not submit data): 

Students achieved an average 5 months of learning beyond expected, which is an outstanding result and similar to the data from the first year of ACoM. Comparative data (comparing the growth of students in this year to previous years) allows us to more confidently conclude that the cause of this strong growth in results was due to the impact of the Alcoa Champions of Maths program through Powerful Problem-Solving (PPS). Fortunately, this year, we received four comparative data sets, all of which strongly support the claim that PPS enhances student learning in mathematics. 

Comparative Case 1: Yr 4 Teacher in ACoM 2021 outperformed all other Yr 4 teachers in the same school who did not do ACoM 

Class 3 achieved 9 months more growth than the average of other Yr 4 classes, and 5 months more than the next best performing class. 

Comparative Case 2: Yr 2 Teacher in ACoM 2021 outperformed all other Yr 2 teachers in the same school who did not do ACoM 

Class 3 achieved 9 months more growth than the average of other Yr 2 classes, and 5 months more than the next best performing class. 

Comparative Case 3: A Yr 4 Teacher from ACoM 2019 taught PPS in 2021 and outperformed all other teachers in Yr 4 in the same school who did not teach PPS. 

The PPS Class (led by ACoM 2019 Teacher) achieved 1 year and 3 months more growth than the average of other Yr 4 classes, and 1 month more than the next best performing class. 

Comparative Case 4: An ACoM 2020 Teacher, retained a cohort of students in 2021, taught PPS again and achieved better results. 

In 2020 we had most classes achieve less growth than expected for a year. One potential explanation was the disruption produced by COVID-19 lockdowns during this year. In 2021 we received some data that supports this explanation. One of the 2020 cohort teachers had some of the same students again in 2021 and applied the same process of teaching maths through Powerful Problem-Solving as had been introduced in 2020. Here are the results: 

The PPS taught students achieved 5 months more growth in 2021 than in 2020. This supports the argument that PPS has a strong positive effect on students’ growth and the poor performance seen in 2020 was due to COVID-19 disruptions (or another effect specific to the year 2020). 

Conclusion 

Average growth of students in PAT-M testing was 6 months ahead of expected in 2019 and 5 months ahead of expected in 2021. Comparison between 2020 and 2021 of students who were in ACoM in 2020, shows they performed ahead of expected when not in 2020, suggesting the disruptions of this year were responsible for the poor performance that year and that PPS still had a powerful positive effect on their learning. Every case in which we have been able to compare the growth of ACoM students to previous years, or to companion classes, they have achieved the greatest growth in the ACoM class/year. A connectionist orientation is known to have a powerful positive effect on student attainment, and all teachers throughout all three years of the program have demonstrated a significant shift toward a more connectionist orientation, averaging between a 13% increase to a 28% increase. All these facts together provide a powerful argument that Powerful Problem-Solving has a significant positive effect on student attainment in mathematics. We are immensely proud to have worked with Alcoa’s support to deliver such a high impact program. 

We intend to keep running a blog called Powerful Problem-Solving which will contain similar content, posts being written by teachers who have previously participated in the Alcoa Champions of Maths program or the Lighthouse Maths program (or participating currently). Stay tuned for more inspiration and ideas from the classrooms of Powerful Problem-Solving practitioners in 2022.