Academic references to outcomes theory

The principle

*If the outcomes performance of units or parties is being compared, a fair comparison of their performance requires either: 1) equality of input; or, 2) effectively adjusting raw outcome measures to compensate for inequality of input. *

The problem

For a variety of purposes, comparisons are often sought between the performance of different units or parties. If they are working on the same inputs (or inputs which are equivalent in terms of the easy of achieving desired outcomes) then it makes sense to just compare raw outcomes. However, where the inputs different groups are working with differ significantly, raw outcomes are a confuse measure of both the value added by a unit or party and the level of input they started with. In these cases, raw outcome is not an accurate measure of performance.

The solution

Where units or parties have inequality of input with which they are working, this needs to be controlled for. It can be controlled for in at least two ways.

1. **The homogeneous group comparison method**. In this method the larger group is broken up into a set of sub-comparisons groups which only compare units or parties with comparable inputs. Raw outcome comparisons are only made between units or parties within these *input-homogeneous* groups.

2. **Statistically controlling for inequality of input**. In this approach, statistical regression techniques are used to adjust raw outcome scores to estimates of what they would have been if there was equality of input across all units or parties.

Examples

Schools differ on the academic level and learning opportunities of students on entry. Therefore a raw comparison of the outcome from schools his unfair in those instances where it is being done between schools or teachers who differ in terms of input. If a feasible *value added* method (that controls for differing input) can be used, it is a better measure of performance than raw outcome measures. Of course, this does not mean that one should not measure raw output for other purposes, just that it should not be used as the measure of teacher or school performance.

In medicine, Casemix (where the length of time (or the dollar value) it takes to treat a certain class of patient (e.g. people under 25, with early onset of schizophrenia) is compared across treatment facilities) is an example of using the first type of solution (*homogenous group comparison*) to this issue.

Value Added measures in regard to teacher performance (which adjust student outcomes on the basis of input level using statistical regression techniques) are an example of the second type of strategy (statistically controlling for inequality of input) solution to this issue.

Resources

See an Op-Ed describing the application of this principle in regard to teacher and school evaluation.