Chasing the McGuffin: Towards the Continuing Reform of Program Evaluation

Abstract

This article focuses its attention on the contemporary need for reform of evaluation and its use – reform in the direction of greater emphasis on localism, on generalizing from small samples and emphasizing direct observation of program interactions. None of these propositions is novel, but it is necessary to reinvent methodological wheels from time to time. As evaluation has become internalized into the administrative system it has been drawn to the business of generalizing beyond and across contexts, seeking to make definitive statements about a program, a domain or a population and assuming that program results are a measure of program quality. This serves the information needs of managers and the policy shaping community – less so those of practitioners whose generalizations are often limited to context. But underlying this is a conflict between distinct forms of knowledge: knowledge for control and knowledge for action. Evaluation is too frequently pressed into the service of the former. A rhetorical device is used to distinguish programme quality as a results measurement from programme quality as the rigorous analysis of programme experience and values. This is Alfred Hitchcock’s ‘McGuffin’ - the objective of a plot which is irrelevant in itself, but which gives meaning to the chase.