Assessing the quality of a proxy indicator 


Academic references to outcomes theory

The principle

Indicator sets that are tidy - in the sense that there might be, for instance 5 outcomes with 5 indicators under each are suspicious. This is because there is no reason to believe that there will be the same number of indicators at a similar level within the outcomes model underlying the indicator set.

Dr Paul Duignan, Three Minute Outcomes video on this principle

The problem

There is a natural tendency to make indicator sets ’tidy’. This is because decision-makers and stakeholders will think that the indicator set looks tidy. A tidy indicator set will also look good on a web-site or in printed format. However there is no theoretical reason to believe that in an instance where there are a number of indicator areas (e.g. conceived of as different outcome areas) that there will be the same number of indicators at the same conceptual level under each indicator.

The solution

As is stated in the princple, it is always best to develop the underlying visual outcomes model for any initiative, organization, sector, community or country before, or at the same time as you are developinmg your indicatgor set. 

Below shows a ’tidy’ indicator set with five outcome areas and five indicators under each of them.

The model below show two outcome areas X and Y. This illustrates the fact that the five indicators under Outcome X are much higher-level than the five indicators under Outcome Y. Just looking at the outcome tidy indicator set above does not provide any information on where the indicators are located within the outcomes model. This approach also means that the reader can identify immediately which boxes are, and are not, being measured.