Measure what matters

Measure what matters


Guest blog post by Chris Lee, more at http://enterpriseessentials.wordpress.com

 

It is better to be approximately right than precisely wrong - source unknown


In a world which is increasingly focused on measuring impact (social, financial and environmental in the world of social enterprise) it's easy to get sucked in to following the crowd.


Yes - it's important to demonstrate how well you're fulfilling your declared purpose/ mission, particularly if you're spending someone else's money. But often the record-keeping is for the (questionable) benefit of others and this can tie you in knots and take you away from the activity you're spending so much time measuring!


We know that most impact is qualitative rather than quantitative and is not easily measured. There's a whole industry around giving such measurement scientific weight, for example through social audits and financial calculations that aim to put a figure on an organisation's social return on investment - SROI. But the people I know who have had this done with their organisations have been disarmingly honest about the dubious reliability of the figures they fed into the process. As they say about computer analysis - 'rubbish in, rubbish out'.


A wonderful bubble-bursting piece by Jeremy Nicholls - a champion and authority on SROI - takes aim at the idea of measuring social impact. He says it assumes we make rational decisions based on evidence (whereas we base them more on preconceptions) and that we have a belief that a shared view is, by definition, correct. Then there are the limitations of predicting the future based on the past.


It's easy to knock the whole process of measurement and counting and many people do (I suspect because they consider it unnecessary extra work) But it may not be a matter of all or nothing.


Squawkpoint blogger James Lawther advises caution when counting (mushy peas is his example) but, in a linked blog, he has suggestions for interpreting the figures once gathered. He recommends looking at the bigger picture - patterns and trends over time - and putting your energy into making sense of the biggest and smallest changes over a period.  And in a seeing-the-wood-for-the-trees attempt at making sense of all the figures available, Lawther recommends using the 80:20 rule to identify which numbers you really need to worry about.     


Many years ago, the New Economics Foundation was influential in my thinking about appropriate measurement. They advocated that you decide what measurement is relevant, meaningful, and achievable in your particular community or organisation.


So if dog mess in a public park is a measure of a community's respect for public spaces, put flags in the poo each month, take photos and compare the numbers of flags month by month. That was in the days before hefty fines and widespread public disapproval - an updated version of this measure might be the number of poo-filled bags left hanging on branches or the number of bags strewn around overflowing bins.


And try not to get pushed into adopting an inappropriate measurement system by funders. In my experience, if you have a system that works for you and present it as such, that can be good enough for them.


Funders don't always know what they mean by 'social impact' (you do, of course!) so if you're ever asked to measure to satisfy outside requirements rather than your own interests, at least be bold enough to ask "why?"


References:


Jeremy Nicholls takes aim at social impact measurement  www.pioneerspost.com/comment/20140404/the-wild-wild-west-of-social-impact

James Lawther on counting mushy peas www.squawkpoint.com/2014/01/operational- and the limits to counting www.squawkpoint.com/2013/06/management-information

For an excellent 'Prove and Improve' toolkit from the New Economics Foundation and Charities Evaluation Service, go to www.proveandimprove.org

ENDS

 

Disclaimer: The views contained in this blog are not necessarily shared by SEEE