There are several keys to asking the right questions when evaluating your program or a new program you may be considering for your center.
You must have:
- A clear understanding of the goals of the program.
- A complete list of activities the program covers.
- Clarity of the context that the program is set in.
You must be able to:
- Identify several evaluation questions related to the five essential process dimensions of effort, performance, process, adequacy, and efficiency.
- Discern the difference between effort from performance or adequacy and understand which is happening in the program setting.
- Understand the difference between dimensions of process and process itself. It is important to understand to what extent program activities are implemented as planned, as well as what changes are made and why before it’s possible to clearly understand how effort and performance should be viewed.
- Prioritize performance and adequacy concerns over efficiency concerns in planning how all issues will be addressed.
- Articulate measurable items to eliminate as much subjectivity as possible from your evaluation.
Here are some tips for developing good evaluation questions:
- Your main evaluation questions should be limited to between 3 and 7. You can create sub-questions for each evaluation question as long as your sub-questions are directly relevant for answering the main question under which they fall.
- Your questions should be prioritized and ranked in order of importance. Questions should refine in order to get at the subjects of greatest importance for the evaluation.
- Your questions should clearly link to the evaluation purpose. Subjects should easily understand what the questions are trying to get from them so that they can provide answers that clearly meet this goal.
- You should be realistic in the number and type of questions created given time and resources available. Questions will not be answered adequately for reaching the evaluation purpose if they are too numerous or complex.
- Your questions should extract definitive answers. Again, Ambiguous or vague answers would be worthless as evaluation data.
- Your questions should reflect real stakeholders’ needs and interests. If questions invoke answers that are complete and thorough yet do not provide data that gives stakeholders the information that they need, it is useless. Questions must extract data that is useable by those who need to use it.
- Your questions should not contain two or more concepts in one. This parallels the requirement for questions to accommodate available time and resources. Such questions are too complex for their context and will extract data that is incomplete, inappropriate to the questions or useless to stakeholders.
- Your questions should focus on how and to what extent something was achieved, forcing complete thoughts as answers instead of simple yes/nos. Articulate questions so that answers require descriptions and quantifications. Nothing of value can be determined by yes/no answers – even if the question concerns a valid concept.