Evaluation can be characterised as being either formative or summative (see Table below). Broadly (and this is not a rule), formative evaluation looks at what leads to an intervention working (the process), whereas summative evaluation looks at the short-term to long-term outcomes of an intervention on the target group.

Formative evaluation

Formative evaluation takes place in the lead up to the project, as well as during the project in order to improve the project design as it is being implemented (continual improvement). Formative evaluation often lends itself to qualitative methods of inquiry.

Summative evaluation

Summative evaluation takes place during and following the project implementation, and is associated with more objective, quantitative methods. The distinction between formative and summative evaluation can become blurred. Generally it is important to know both how an intervention works, as well as if it worked. It is therefore important to capture and assess both qualitative and quantitative data.

Types of evaluation

Source: Owen & Rogers (1999)

Goals-based evaluation

Evaluation has typically involved measuring whether predetermined targets have been met. You may be familiar with the term ‘SMART’ (Specific, Measurable, Attainable, Relevant, and Timely) targets. These type of targets fall under the banner of goals-based evaluation.

Goals-based evaluations have objectively-set targets usually determined by people responsible for the funding or implementation of the project.  Goals-based evaluation does not question whether the selected goals are valid, nor whether appropriate measures of effectiveness are being assessed.
Recent behaviour change and research and evaluation practice proposes the use of goals-free evaluation.

Goals-free evaluation

Goals-free evaluation looks at emergence and unintended consequences of a project, by looking at the actual effects without pre-empting what these may be. This type of evaluation focuses on the change process within the target group. The goals-free evaluation parallels a social learning approach to evaluation which can be desirable where the goal of a project seeks to build capacity of people to manage a complex issue. In such cases, relying solely on a goals-based evaluation may not be appropriate as it can be difficult to set targets and quantify a process of change, as well as capture change which one may not anticipate.

A key point to consider in setting goals and indicators of success is the validity of the indicator. This is especially important when you are looking to measure a largely “unobservable” behaviour (for example, those that happen within the home, and cannot be readily objectively observed). In such cases, proxy indicators are often used (for example, household electricity use) but these indicators may be gross measures that consist of many varied behaviours, of which only one may be of interest. The question is how to establish meaningful indicators?