What is a program evaluation?
The goals of program evaluations are to make judgments about program effectiveness, suggest ways for program improvement, and recommend to decision makers future programming directions. Program evaluations are based on information is that collected systematically. Some of the information may be quantitative (numerical and statistical), and some qualitative (observational, interviews, documents).
What is an outcome evaluation?
Outcome evaluations focus on the immediate effects of the program on participants. Outcome evaluations almost always use quantitative or statistical data from which to draw conclusions. For example, in a program designed to prevent teen smoking through teaching physical fitness concepts, an outcome evaluation would focus on whether teens smoke less after experiencing the program. A related outcome question would ask whether the participants are more physically fit after going through the program.
What is an implementation evaluation?
Implementation evaluations focus on what really happened when the program was put in the field. Most programs take different shapes and forms when they reach the field. Implementation evaluations collect information at the 'ground level', in order to see how different components of the program were modified in special situations. Such studies often lead to 'best practice models.' For example, one statewide program was designed to prevent teen smoking by funding ethnic and cultural minority groups to design culturally relevant anti-tobacco materials. The implementation evaluation was able to document some of the most interesting approaches taken in the field:
What is a process evaluation?
Process evaluations look at program activities -- in particular, the flow of information and resources in a program. Process evaluations must seek to describe how the funding agency interacts with, shares information with, and distributes resources to the local program sites.For example, in a program designed to prevent teen smoking through teaching physical fitness concepts, the process evaluation covered the whole life of the project - not just when it made it to the schools. The evaluators observed how the project directors selected among applicants; the evaluators attended training workshops for the teachers who were awarded the project; and they visited sites to see how central project staff supported the program.
How should I choose an evaluator for my project?
The most important step is to read evaluation reports produced by each competing firm. See whether they appear clear, non-jargonistic, supported by evidence. See whether they would appear to be useful for their target audience: funding agencies, project staff, decision-makers. See what parts seem especially relevant to your particular project needs. See if the evaluation lends itself to helping to improve the program - not just making summary statements about its effectiveness. See if you get a deep understanding of what the project actually looked like in the field.
If at all possible, it is important to meet with the evaluation team before making your decision. Try to determine what it will be like to work together toward program improvement. Sit down and explain the goals of the project to the evaluators, and see if they appear to understand them.
How much should an evaluation cost?
Cost varies considerably depending on what the evaluation consists of: sampling of participants, visits to sites, instrument design, etc. But a good rule of thumb is to try and commit 15 percent of the overall project cost to evaluation.
|
