Monitoring
Monitoring is defined in the Social Way as “the ongoing collection and analysis of financial and non-financial information to check performance against stated objectives, budget and work plan. Monitoring is normally concerned with inputs, activities and outputs.” Monitoring presents what has been delivered and answers the question: did we do what we said we would do? In simple terms the purpose of doing monitoring is to measure if we have implemented the activities (e.g. project or risk / impact controls) that we had planned by measuring against defined indicators.
Monitoring requires defining, in advance, the target indicators for inputs, and outputs of an intervention (e.g. project or risk / impact controls). For example:
- To monitor an project related to capacity development, sites will need to define and measures progress against:
- Inputs indicator targets: a budget of $25,000, 0.5 full-time employees dedicated to the implementing the project for two months.
- Outputs indicator targets: 50 people to participate in the training, five separate training sessions to be delivered.
- To monitor an impact control related to mitigating the potential for vehicle accidents involving site vehicle in the community, sites will need to define and measures progress against:
- Inputs indicator targets: a budget of $10,000, 0.1 full-time employees dedicated to setting up the control for one month.
- Outputs indicator targets: 15 vehicle warning signs erected, five sets of ‘speed bump’ calming measures constructed.
Monitoring tends to focus on progress at any given point of time against targets. Monitoring might tell us that a problem exists i.e. that an indicator is far below target, but will not give the depth of information to understand why the problem has occurred, and how it could be addressed. Monitoring is an ongoing process (e.g. monthly or quarterly) and typically undertaken by Anglo American staff and/or staff working at an external organisation in the case of an project implementer.
Evaluation
Under the Social Way, evaluation is understood as “the process of determining the outcomes of an intervention or programme, as well as its associated value for the intended beneficiaries, on the basis of systematic and regular data collection (monitoring).” Its purpose is to improve the effectiveness and sustainability of interventions (e.g. project or risk / impact controls) to inform decision-making about future plans. Evaluation goes beyond monitoring to answer the question: what has happened as a result of our activities, and has the stated objective been met?
Evaluation requires:
- Analysis regarding why the intended objectives were or were not achieved;
- An assessment of which factors contributed or constrained achievement of the intended objectives;
- Consideration of what the consequences of the activity have been, both intended and unintended; and
- Documentation of information regarding successes, challenges, whether objectives were met or not and areas for improvement.
While monitoring focuses on tracking progress in implementation, evaluation focuses predominantly on effectiveness, sustainability and replicability i.e. have we managed to attain the results expected; will the intervention bring long-term value; and does the intervention merit repeating (on a cost-benefit basis as well as in terms of its success in achieving the desired results)? Evaluation requires defining, in advance, the target indicators for outcomes and the impact of the intervention (e.g. project or risk / impact controls). For example:
- To evaluate an project related to capacity development, sites will need to define and measures progress against:
- Outcome indicator targets: 30 stakeholders use the skills they gained to register their business with supply chain, 80% of participants indicate they have achieved the learning objectives of the training.
- Impact indicator targets: Increasing percentage of local procurement delivered to local businesses, increase in average household income in targeted area (against baseline).
- To evaluate an impact control related to mitigating the potential for vehicle accidents involving site vehicles in the community, sites will need to define and measures progress against:
- Outcome indicator targets: reduction in relevant ‘near miss’ potential incidents recorded (compared to baseline), reduced average vehicle speed passing through relevant area (against baseline).
- Impact indicator targets: 0 vehicle accidents resulting in injury to stakeholders involving site vehicle in the community.
Evaluation should be conducted at periodic intervals as relevant for the scale and duration of the activity. In the case of longer term activities an interim evaluation should be conducted as relevant (ensuring major flaws or problems can be addressed) with a more comprehensive evaluation undertaken on completion (informing the for the forthcoming year).
For specific time-bound projects (e.g. a three-year partnership to deliver skills training), a mid-term evaluation should be conducted at 18 months (or as relevant) to identify any corrective actions to improve achievement of the stated objective. A full evaluation could then be completed at the end of the programme for the purpose of assessing if the activity has adequately achieved the stated outcomes and impact. Evaluations are often completed by external organisations to bring objectivity and independence.
Figure 1.1 illustrates the different areas of focus between monitoring and evaluation.
Figure 1.1 Focus areas of monitoring and evaluation