Why Conduct an Evaluation?
There are two simple reasons for conducting an evaluation:
• To gain direction for improving projects as they are developing, and
• To determine projects’ effectiveness after they have had time to produce results.
Formative evaluations (which include implementation and process evaluations) address the first set of issues. They examine the development of the project and may lead to changes in the way the project is structured and carried out. Questions typically asked include:
• To what extent do the activities and strategies match those described in the plan? If they do not match, are the changes in the activities justified and described?
• To what extent were the activities conducted according to the proposed timeline? By the appropriate personnel?
• To what extent are the actual costs of project implementation in line with initial budget expectations?
• To what extent are the participants moving toward the anticipated goals of the project?
• Which of the activities or strategies are aiding the participants to move toward the goals?
• What barriers were encountered? How and to what extent were they overcome?
Summative evaluations (also called outcome or impact evaluations) address the second set of issues. They look at what a project has actually accomplished in terms of its stated goals. Summative evaluation questions include:
• To what extent did the project meet its overall goals?
• Was the project equally effective for all participants?
• What components were the most effective?
• What significant unintended impacts did the project have?
• Is the project replicable and transportable?
For each of these questions, both quantitative data (data expressed in numbers) and qualitative data (data expressed in narratives or words) can be useful in a variety of ways.
The remainder of this chapter provides some background on the differing and complementary nature of quantitative and qualitative evaluation methodologies. The aim is to provide an overview of the advantages and disadvantages of each, as well as an idea of some of the more controversial issues concerning their use.
Before doing so, however, it is important to stress that there are many ways of performing project evaluations, and that there is no recipe or formula that is best for every case. Quantitative and qualitative methods each have advantages and drawbacks when it comes to an evaluation’s design, implementation, findings, conclusions, and utilization. The challenge is to find a judicious balance in any particular situation. According to Cronbach (1982), There is no single best plan for an evaluation, not even for an inquiry into a particular program at a particular time, with a particular budget.
What Are the Major Differences Between Quantitative and Qualitative Techniques?
As shown in Exhibit 1, quantitative and qualitative measures are characterized by different techniques for data collection.
Exhibit 1. Common techniques
(Sursa: User-Friendly Handbook for Mixed Method Evaluations)