John Hattie has synthesized more than 800 meta-studies related to achievement. In his book Visible Learning he found 138 influences with positive and negative effects on learning outcomes. Some of Hattie’s critics state that comparing so many different things would be like comparing apples and oranges. But is it? The following visualizations show how many studies Hattie actually read and synthesized for his ground-breaking meta-meta-study on what works in the classroom.
How many meta-studies has John Hattie synthesized for each of the 138 influences?
The picture shows a relatively balanced distribution of the 800+ meta-studies with an emphasis on “Computer assisted instruction”, “Gender” and “Feedback”.
How many underlying studies has John Hattie synthesized for each of the 138 influences?
The distribution of the studies underlying the 138 influences is a bit less balanced. (The 800+ meta-studies synthesized by John Hattie are based on 50.000+ studies.) In the underlying studies you can see a strong focus on “Teaching strategies”, “Computer assisted instruction”, “Prior achievement” and “Gender”.
What is the underlying number of people studied for each of the 138 influences?
More than 80 million people have been studied in the research John Hattie synthesized for Visible Learning. The number of people for each influence are quite unevenly distributed across the 138 factors. Half of the people belong to research on “Comprehensive teaching reforms” (41.9m). Much less people were studied in research on “Family structure” (10.1m), “Drama/Arts programs” (5.8m), “Gender” (5.6m) and “Distance education” (4m). Moreover, the visualization takes into account the fact that John Hattie did only report the number of people for 98 out of 138 influences, 40 influences can not be visualized.
Conclusion
Some quick take-aways of these visualizations:
- Not all of the 138 influences are studied to the same (quantitative) extent. More particularly John Hattie did not study 80 million people for each of the 138 influences.
- More than 60 million people belong to studies related to influences on learning that are not very effective (according to Hattie’s standards). His top 10 influences synthesize studies with less than 1 million people.
- No numbers are reported for 40 influences, including 3 influences of Hattie’s top 10.
Hattie definitely doesn’t compare apples with oranges. But one could say: The studied fruits belong to baskets of quite different sizes.
I think it is really interesting how John Hattie has conducted all these studies to help teachers better influence their students in learning more effectively through feedback.