What is the true intention of measuring outcomes of interventions? What are we trying to prove? or are we trying to learn as much as we can from our practice.
One of the most insidious and damaging movements I have seen in the public sector over the past 25 years has been the use of outcome measures as performance rather than effectiveness tools. There appears to be a general belief that if services, professions, organisations could just find a measure that could prove what they are doing works, other people will recognise their work and allow them to get on with it.
I have always believed that professionals and the public bodies have a duty to know whether their contribution is making a real difference to the populations they serve. However, I see far too many examples of “progress” being used to either prove or disprove worth or lack of it. Reflective conversations about what really works have been replaced with targets. Outcome measurement as a quality tool has become another by-industry of Health, Education and Social Care services. It has made a fortune from the deluded idea that the measure is king! Just find the right measure and we will all be able to compare services and practitioners, identify the “outstandingly good” ones and weed out the “failing” ones.
Learning not Proving!
Surely the key is the intention and the meaning that is derived from measuring outcome. In other words, once a practitioner understands what needs to and can change now for a given person, family or community, s/he can know what measure would be most effective in ascertaining whether change has happened.
Outcome achieved does not necessarily mean that the practitioner who achieved it is better or worse than the one whose predicted outcome was not achieved. Performance should not be measured by a high number of outcomes achieved. Whilst this it obviously desirable, it is the learning that is gained from reflecting on the outcome, and its bearing on the next hypothesis/formulation, that should inform performance judgements. In other words, evidence of reflective practice.
Health, Social Care and Education practitioners can only really evidence their worth by evidencing that they are working hard to support people to find a solution that works for them. The person’s meaning drives the hypothesis about whether they can or cannot be helped by the services they approach. Outcome simply indicates whether they have been helped once a course of action has been agreed.
The intention will change both in relation to who needs the help and what is possible. Therefore, the measure will change in parallel. If a teacher predicts that a child needs to feel more confident in her understanding of numbers before she can do algebra, the measure she would use after trying to boost her confidence, would need to be a measure of confidence and ease with numbers! If she used an algebra test result at this stage, she could, in fact, reduce the child’s confidence and make it less likely she would pass algebra!Most of the outcomes measures on the market these days have their place – often rigorously tested to ensure reliability and validity. However their true validity relates to the practitioners skill in choosing the measure that fits the intention and not making the intention fit the measure.