Our Kids Are Not Broken

It is a beautiful day here in Melbourne. I hope you are well and your week has been fun. What have you been grateful for?

Thanks for exploring today’s issue. Three connected ideas, it is up to you to find your thread. 🪡🧵

The Dialogic Learning Weekly - Revue

Do we measure what we value?

When Measurement Collapses

Goodhart’s Law is a mental model I am sharing to add to your cognitive toolkit. The ideas were first shared by Charles Goodhart, a leader in monetary policy, at a conference in Sydney, Australia, in 1975.

During a presentation, he said:

“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”

Rephrased as:

“When a measure becomes a target, it ceases to be a good measure.”

According to Peter Coy, it was Marilyn Strathern who coined this more accessible version.

It got me thinking about assessment, monitoring and evaluation in education. So many elements of our system(s) relate to this mental model and its contradiction. What springs to mind for you?

From the Peter Coy article, I also love the following example of Goodhart’s Law:

when Soviet planners ordered nail factories to increase the number of nails they produced, it’s said that managers reacted by producing millions of tiny, useless nails. When the planners wised up and switched to a weight criterion, the factories started producing giant, heavy, and equally useless nails. It’s hard to pin down the historical truth of these stories, but the point is clear.

Coy provides a range of other examples from modern life and explains that simply being aware of the pitfalls of measurement is a good step.

I know I will be using this mental model to think about assessment and review feedback.

get away from fixed, changeless rules that can be easily gamed. Also, measure what you actually want, not a rough proxy for it. Try using multiple criteria instead of a single standard. But the first step is simply to be aware of the problem. Putting a name and a focus on the pitfalls of measurement and reward is the lasting contribution of Charles Goodhart.

Goodhart’s Law Rules the Modern World. Here Are Nine Examples — Bloomberg — www.bloomberg.com

Our Kids Are Not Broken

I worked with teachers in five or six different countries this week. The difference between realities in school at the moment is stark. Stories of the challenges of returning to a version of ‘school’ fill my network — will it ever look the same?

In The Atlantic, (publication not ocean) Ron Berger’s response to the back on track or learning loss narrative surfacing in many countries resonated with me. I will let his words sing through:

The pressure to measure — and remediate — this “learning loss” is intense; many advocates for educational equity are rightly focused on getting students back on track. But I am concerned about how this growing narrative of loss will affect our students, emotionally and academically.

If districts focus too much on remediating “learning loss” — holding kids back a grade, categorizing students according to their deficits, and centering lesson plans on catch-up work — the students who have experienced the most trauma and disconnection during the pandemic may be assigned to the lowest level and most stigmatized groups.

But our kids are not broken. To foster students’ growth, districts should think beyond traditional ways of grading and teaching. Instead of federal and district test results becoming labels, handed down as if from on high, districts should use them diagnostically, as guides only, and encourage teachers to collaborate with students in understanding their skill profiles so that the kids feel empowered in their own development.

There is so much to reflect on, and I encourage you to explore the entire opinion piece. One additional highlight for us to explore is a brief remark about an alternative path. Notice the subtleties in phrasing.

educators need to assess students’ abilities in a way that motivates them to grow.

From which we can derive the problem statement:

How might we assess students’ abilities in a way that motivates them to grow?

Our Kids Are Not Broken

Numbers Without Narratives

The complex work of education occurs in an equally complex system. (perhaps a circular reference) It seems my perspective is frequently drawn to the system level these days. Zooming out to a thousand feet view is a powerful method.

What are the guiding principles for a system to design measures of progress and impact?

I enjoyed some of the insight fragments shared by the team at Complexability:

Numbers without narratives — the what without the why provides no basis for action. The lack of context promotes speculation, favours biased choices of ‘answers’ and promises of solutions that have little substance.

In the short post below, they propose a range of elements we might consider to design better measurement systems. As educational assessment and monitoring processes occur with inherent complexity, they may be relevant.

Effective monitoring and feedback processes:

• address the whole system,
• are concerned with impact over time,
• do not attempt to attribute cause and effect,
• develop quantitative and qualitative measures are appropriate, and meaningful to those who need to engage and report.

I have picked a few that resonate with me — I need to ponder the cause and effect one — please read the rest in the blog post below.

Measuring and Monitoring Impact... | complexability

I am grateful to Jeff Sheldon for the wooden ruler photo. It reminded me of the vast array of random rulers large and small we had when I was teaching.

What thread did you see? Which provocation will you share with a colleague?

Let me know what resonates.

In dialogue we trust.

~ Tom