7 Types of Evaluation You Need to Know

Development programs have to prove that they have had a strong and positive impact. Different types of evaluations are invaluable in showing the benefits of a program to backers, sponsors, and program beneficiaries.

In the past, measuring development programs involved pen-and-paper surveys and many people. This was inefficient, expensive and time consuming, and at times it resulted in inaccurate data. The results would often only be available after months.

Today, thanks to improvements in technology, monitoring and evaluation (M&E) has become digital. This has increased efficiency and accuracy while reducing costs, making it easier to conduct different types of evaluation at different stages of a program. Monitoring and evaluation personnel can now communicate the effectiveness of their program to partners, sponsors, program officers, and the community at large.

The right types of evaluation for you

We’ve put together 7 types of evaluation that you need to know about to have an effective M&E system. Using these types of evaluation can help your program deliver better results and have a greater impact, all while reducing costs!

Choosing the best types of evaluation depends on the stage at which your development program is. Each evaluation can help you make better decisions by giving you the right kind of data at the right time.

The best development project will conduct different types of evaluations, constantly looking to streamline their project or program at different stages and using different metrics.

Formative Evaluation

(also known as ‘evaluability assessment’)

Formative evaluation is used before program design or implementation. It generates data on the need for the program and develops the baseline for subsequent monitoring. It also identifies areas of improvement and can give insights on what the program’s priorities should be. This helps project managers determine their areas of concern and focus, and increases awareness of your program among the target population prior to launch.

When:

• New program development
• Program expansion

What:

• The need for your project among the potential beneficiaries
• The current baseline of relevant indicators, which can help show impact later

Why:

• Helps make early improvements to the program
• Allows project managers to refine or improve the program

How:

Conduct sample surveys and focus group discussions among the target population focused on whether they are likely to need, understand, and accept program elements.

Questions to ask:

• Is there a need for the program?
• What can do to improve it?

Process Evaluation

(also known as ‘program monitoring’)

Process evaluation occurs once program implementation has begun, and it measures how effective your program’s procedures are. The data it generates is useful in identifying inefficiencies and streamlining processes, and portrays the program’s status to external parties.

When:

• When program implementation begins
• During operation of an existing program

What:

• Whether program goals and strategies are working as they should
• Whether the program is reaching its target population, and what they think about it

Why:

• Provides an opportunity to avoid problems by spotting them early
• Allows program administrators to determine how well the program is working

How:

Conduct a review of internal reports and a survey of program managers and a sample of the target population. The aim should be to measure the number of participants, how long they have to wait to receive benefits, and what their experience has been.

Questions to ask:

• Who is being reached by the program?
• How the program is being implemented and what are the gaps? Is it meeting targets?

Outcome Evaluation

(also known as ‘objective-based evaluation’)

Outcome evaluation is conventionally used during program implementation. It generates data on the program’s outcomes and to what degree those outcomes are attributable to the program itself. It is useful in measuring how effective your program has been and helps make it more effective in terms of delivering the intended benefits.

When:

• After the program has run for some time period
• At an appropriate time to measure outcomes against set targets – usually benchmarked time periods

What:

• How much the program has affected the target population
• Clearly establish the degree of benefit provided by the program

Why:

• Helps program administrators tell whether a program is meeting its objectives
• Insights from outcome-focused feedback can help increase effectiveness

How:

A randomized controlled trial, comparing the status of beneficiaries before and during the program or comparing beneficiaries to similar people outside of the program. This can be done through a survey or a focus group discussion.

Questions to ask:

• Did participants report the desired change after the implementation of the program?
• What are the short or long-term results reported by participants?

Economic Evaluation

(also known as ‘cost analysis’, ‘cost-effectiveness evaluation’, ‘cost-benefit analysis’, and ‘cost-utility analysis’)

Economic evaluation is used during the program’s implementation and looks to measure the benefits of the programs against the costs. Doing so generates useful quantitative data that measures the efficiency of the program. This data is like an audit, and provides useful information to sponsors and backers who often want to see what benefits their money would bring to beneficiaries.

When:

• At the beginning of a program, to remove potential leakages
• During the operation of a program, to find and remove inefficiencies.

What:

• What resources are being spent and where
• How these costs are translating into outcomes

Why:

• Program managers and funders can justify or streamline costs
• The program can be modified to deliver more results at lower costs

How:

A systematic analysis of the program by collecting data on program costs, including capital and man-hours of work. It will also require a survey of program officers and the target population to determine potential areas of waste.

Questions to ask:

• Where is the program spending its resources?
• What are the resulting outcomes?

Impact Evaluation

Impact evaluation studies the entire program from beginning to end (or at whatever stage the program is at), and looks to quantify whether or not it has been successful. Focused on the long-term impact, impact evaluation is useful for measuring sustained changes brought about by the program or making policy changes or modifications to the program.

When:

• At the end of the program
• At pre-selected intervals in the program

What:

• Assesses the change in the target population’s well-being
• Accounts for what would have happened if there had been no program

Why:

• To show proof of impact by comparing beneficiaries with control groups
• Provides insights to help in making policy and funding decisions

How:

A macroscopic review of the program, coupled with an extensive survey of program participants, to determine the effort involved and the impact achieved. Insights from program officers and suggestions from program participants are also useful, and a control group of non-participants for comparison is helpful.

Questions to ask:

• What changes in program participants’ lives are attributable to your program?
• What would those not participating in the program have missed out on?

Summative Evaluation

Summative evaluation is conducted after the program’s completion or at the end of a program cycle. It generates data about how well the project delivered benefits to the target population. It is useful for program administrators to justify the project, show what they have achieved, and lobby for project continuation or expansion.

When:

• At the end of a program
• At the end of a program cycle

What:

• How effectively the program made the desired change happen
• How the program changed the lives of program participants

Why:

• Provides data to justify continuing the program
• Generates insights into the effectiveness and efficiency of the program

How:

Conduct a review of internal reports and a survey for program managers and target populations. The aim should be to measure the change that the project has brought about and compare the change to the costs.

Questions to ask:

• Should the program continue to be funded?
• Should the program be expanded? If so, where? What factors worked in its favor and what worked against it?

Goals-Based Evaluation

(also known as ‘objectively set evaluation)

Goals-based evaluation is usually done towards the end of the program or at previously agreed-upon intervals. Development programs often set ‘SMART’ targets — Specific, Measurable, Attainable, Relevant, and Timely — and goals-based evaluation measures progress towards these targets. The evaluation is useful in presenting reports to program administrators and backers, as it provides them the information that was agreed upon at the start of the program.

When:

• At the end of the program
• At pre-decided milestones

What:

• How the program has performed on initial metrics
• Whether the program has achieved its goals

Why:

• To show that the program is meeting its initial benchmarks
• To review the program and its progress

How:

This depends entirely on the goals that were agreed upon. Usually, goals-based evaluation would involve some survey of the participants to measure impact, as well as a review of input costs and efficiency.

Questions to ask:

• Has the program met its goals?
• Were the goals and objectives achieved due to the program or externalities?

Development programs with effective monitoring and evaluation frameworks use different types of evaluation at different points of time. Some programs might even run two different types of evaluation at the same time for entirely different purposes. No matter what types of evaluation you use, we hope you find this blog useful in making your project and program more successful and efficient!