Development programs have to prove that they have had a strong and positive impact. Different types of evaluations are invaluable in showing the benefits of a program to backers, sponsors, and program beneficiaries.
In the past, measuring development programs involved pen-and-paper surveys and many people. This was inefficient, expensive and time consuming, and at times it resulted in inaccurate data. The results would often only be available after months.
Today, thanks to improvements in technology, monitoring and evaluation (M&E) has become digital. This has increased efficiency and accuracy while reducing costs, making it easier to conduct different types of evaluation at different stages of a program. Monitoring and evaluation personnel can now communicate the effectiveness of their program to partners, sponsors, program officers, and the community at large.
The right types of evaluation for you
We’ve put together 7 types of evaluation that you need to know about to have an effective M&E system. Using these types of evaluation can help your program deliver better results and have a greater impact, all while reducing costs!
Choosing the best types of evaluation depends on the stage at which your development program is. Each evaluation can help you make better decisions by giving you the right kind of data at the right time.
|Stage of Project||Purpose||Types of Evaluation|
|Conceptualization Phase||Helps prevent waste and identify potential areas of concerns while increasing chances of success.|
|Implementation Phase||Optimizes the project, measures its ability to meet targets, and suggest improvements for improving efficiency.|
|Project Closure Phase||Insights into the project’s success and impact, and highlight potential improvements for subsequent projects.|
The best development project will conduct different types of evaluations, constantly looking to streamline their project or program at different stages and using different metrics.
(also known as ‘evaluability assessment’)
Formative evaluation is used before program design or implementation. It generates data on the need for the program and develops the baseline for subsequent monitoring. It also identifies areas of improvement and can give insights on what the program’s priorities should be. This helps project managers determine their areas of concern and focus, and increases awareness of your program among the target population prior to launch.
- New program development
- Program expansion
- The need for your project among the potential beneficiaries
- The current baseline of relevant indicators, which can help show impact later
- Helps make early improvements to the program
- Allows project managers to refine or improve the program
Conduct sample surveys and focus group discussions among the target population focused on whether they are likely to need, understand, and accept program elements.
Questions to ask:
- Is there a need for the program?
- What can do to improve it?
(also known as ‘program monitoring’)
Process evaluation occurs once program implementation has begun, and it measures how effective your program’s procedures are. The data it generates is useful in identifying inefficiencies and streamlining processes, and portrays the program’s status to external parties.
- When program implementation begins
- During operation of an existing program
- Whether program goals and strategies are working as they should
- Whether the program is reaching its target population, and what they think about it
- Provides an opportunity to avoid problems by spotting them early
- Allows program administrators to determine how well the program is working
Conduct a review of internal reports and a survey of program managers and a sample of the target population. The aim should be to measure the number of participants, how long they have to wait to receive benefits, and what their experience has been.
Questions to ask:
- Who is being reached by the program?
- How the program is being implemented and what are the gaps? Is it meeting targets?
(also known as ‘objective-based evaluation’)
Outcome evaluation is conventionally used during program implementation. It generates data on the program’s outcomes and to what degree those outcomes are attributable to the program itself. It is useful in measuring how effective your program has been and helps make it more effective in terms of delivering the intended benefits.
- After the program has run for some time period
- At an appropriate time to measure outcomes against set targets – usually benchmarked time periods
- How much the program has affected the target population
- Clearly establish the degree of benefit provided by the program
- Helps program administrators tell whether a program is meeting its objectives
- Insights from outcome-focused feedback can help increase effectiveness
A randomized controlled trial, comparing the status of beneficiaries before and during the program or comparing beneficiaries to similar people outside of the program. This can be done through a survey or a focus group discussion.
Questions to ask:
- Did participants report the desired change after the implementation of the program?
- What are the short or long-term results reported by participants?
(also known as ‘cost analysis’, ‘cost-effectiveness evaluation’, ‘cost-benefit analysis’, and ‘cost-utility analysis’)
Economic evaluation is used during the program’s implementation and looks to measure the benefits of the programs against the costs. Doing so generates useful quantitative data that measures the efficiency of the program. This data is like an audit, and provides useful information to sponsors and backers who often want to see what benefits their money would bring to beneficiaries.
- At the beginning of a program, to remove potential leakages
- During the operation of a program, to find and remove inefficiencies.
- What resources are being spent and where
- How these costs are translating into outcomes
- Program managers and funders can justify or streamline costs
- The program can be modified to deliver more results at lower costs
A systematic analysis of the program by collecting data on program costs, including capital and man-hours of work. It will also require a survey of program officers and the target population to determine potential areas of waste.
Questions to ask:
- Where is the program spending its resources?
- What are the resulting outcomes?
Impact evaluation studies the entire program from beginning to end (or at whatever stage the program is at), and looks to quantify whether or not it has been successful. Focused on the long-term impact, impact evaluation is useful for measuring sustained changes brought about by the program or making policy changes or modifications to the program.
- At the end of the program
- At pre-selected intervals in the program
- Assesses the change in the target population’s well-being
- Accounts for what would have happened if there had been no program
- To show proof of impact by comparing beneficiaries with control groups
- Provides insights to help in making policy and funding decisions
A macroscopic review of the program, coupled with an extensive survey of program participants, to determine the effort involved and the impact achieved. Insights from program officers and suggestions from program participants are also useful, and a control group of non-participants for comparison is helpful.
Questions to ask:
- What changes in program participants’ lives are attributable to your program?
- What would those not participating in the program have missed out on?
Summative evaluation is conducted after the program’s completion or at the end of a program cycle. It generates data about how well the project delivered benefits to the target population. It is useful for program administrators to justify the project, show what they have achieved, and lobby for project continuation or expansion.
- At the end of a program
- At the end of a program cycle
- How effectively the program made the desired change happen
- How the program changed the lives of program participants
- Provides data to justify continuing the program
- Generates insights into the effectiveness and efficiency of the program
Conduct a review of internal reports and a survey for program managers and target populations. The aim should be to measure the change that the project has brought about and compare the change to the costs.
Questions to ask:
- Should the program continue to be funded?
- Should the program be expanded? If so, where? What factors worked in its favor and what worked against it?
(also known as ‘objectively set evaluation)
Goals-based evaluation is usually done towards the end of the program or at previously agreed-upon intervals. Development programs often set ‘SMART’ targets — Specific, Measurable, Attainable, Relevant, and Timely — and goals-based evaluation measures progress towards these targets. The evaluation is useful in presenting reports to program administrators and backers, as it provides them the information that was agreed upon at the start of the program.
- At the end of the program
- At pre-decided milestones
- How the program has performed on initial metrics
- Whether the program has achieved its goals
- To show that the program is meeting its initial benchmarks
- To review the program and its progress
This depends entirely on the goals that were agreed upon. Usually, goals-based evaluation would involve some survey of the participants to measure impact, as well as a review of input costs and efficiency.
Questions to ask:
- Has the program met its goals?
- Were the goals and objectives achieved due to the program or externalities?
Development programs with effective monitoring and evaluation frameworks use different types of evaluation at different points of time. Some programs might even run two different types of evaluation at the same time for entirely different purposes. No matter what types of evaluation you use, we hope you find this blog useful in making your project and program more successful and efficient!
Was very informative; Easy to understand.
Thanks for the insights but would be more rich if an examples of how it looks would be great for more learn as many amy not have the idea of how it looks and the design of the plan.what constitues m/e plan .any diference with m/e framworks.Thank you so much.please provide a draft example of how it looks and designed.
Thanks, I’ll add it to our list of ideas. Stay tuned to our blog.
This is a really insightful post! Though it would have been more enriching if it was supported by an example of how these methods fit into a real-time program application.
Thanks Toral, that’s a great point. I’ll add that to our list of future topics.
I am sure many will find this useful. Please note that evaluation guru Michael Quinn Patton lists 80 types of evaluation in his book Utilization-Focused Evaluation, 4th ed (Patton, 2008), Exhibit 8.1, pp. 300-305. This book is well worth reading – one of the classics in the field.
Thanks. Content quit useful and practical in most field cases.
If formative evaluation is taken place during the Conceptualization Phase, can it be called also as ” Ex ante” evaluation ?
Good question, Suthan. Technically, yes — since formative evaluation happens before the program kicks off, it happens “ex ante” (before the event). Still, it’s probably better to stick with the term formative evaluation.
Ex-ante evaluation is actually its own type of evaluation. Unfortunately, “ex-ante evaluation” is used to describe lots of different evaluation activities that happen before a program starts — e.g. formative assessment, impact assessment, priority setting, budgeting, baseline surveys, and more. That’s part of why we didn’t include ex-ante evaluation in this list.
Would you please let me know what other topics you have in your Blog
Thanks, this is concise and clear.
I’d add (ex-post- Sustained and Emerging Impacts Evaluation: betterevaluation.org/en/themes/SEIE
thanks this is short precise and clear
Thank you very much. This has been very useful and resourceful. Printable formats will also be appreciated.
Hi Dr. Emmanuel, thanks for the feedback. We’ve considered adding PDF versions of our blogs for a while, but we never spent the time on it because we weren’t sure if people would want them. It’s good to know you’d find it useful!
I agree with you.
Same amount of energy can be used for another topic.
If you kindly explain about Type of objectives just like types of evaluation explained!!!
Really informative information, short and clear to understand. Thank you.
I am still not much clear in the impact evaluation as it categories in formative evaluation types, am I right? Isn’t it should be in summative evalution
You’re right that impact evaluation and summative evaluation are similar.
One key difference is that impact evaluation can happen any time (during or after), while summative evaluation happens at the end of a program or program cycle. Also, impact evaluation also usually focuses on how the population benefited from the program (in comparison to a control group), while summative evaluation focuses on comparing the program’s benefits to its costs.
Overall, they’re similar and are both ways to look at how a program is doing. They just have slightly different objectives and timings. Does that help?
P.S. I’m from SocialCops and I helped out with this blog, so I’d love to know if it’s clear and helpful or not. Please let me know!
Concise and easy to understand! Many thanks for sharing.
thanks to your rigorous, insightful information and astonishment hard working, I learned useful information from your post but unfortunately I did not understand when what why, and how what does it mean, kindly explain a little bit more.
Thank you for putting this information for us.
However, I would like to understand the difference between formative evaluation and baseline survey/study. I always get confused between the two. I would greatly appreciate your help.
Thanks once again.
Clear and good to understand the different types. But some evaluations seem to almost answer the same question. Is it necessary to carry out all these different types given the scarse resources?
Your work is excellent, I have enjoyed every bit of it. Can I have a PDF or printer copy.
very insightful,i shall be using this in field work thank you.
Waoo its understandable
My comment is that I want to know about this course because it is my major that I reading presentely.
Very perfect research carried out,
Thanks for the training and education that makes me to understand some of monitoring and evaluation because it is my area of studies at same time works as an M&E personel at Monrovia city corporation mcc in Liberia
I really appreciate the notes are very good and understandable
very good and summarized