How do you know if you’re getting the best quality in your evaluations?
Quality in evaluation used to be defined as rigor (and sometimes still is), with rigor meaning the competence of the evaluator, the legitimacy of the process and, of course, applying the best research methods to the collection and analysis of data. These are important, but they don’t count as an all-encompassing definition of quality, particularly in complex, adaptive settings where evaluation partners with strategy.
If we cannot count of these measures to define quality, what are alternative ways of understanding if your evaluation is high quality? Hallie Preskill from FSG and I will be joining forces at the American Evaluation Association’s annual conference this Friday to explore this issue. We are proposing that the concept of “rigor” (and thus what you can look for in your evaluations) can – and should – be redefined as:
- Balancing whether the evaluation is useful, inclusive of multiple perspectives, unbiased, accurate, and timely.
- The quality of the learning process, including whether it engages the people who need the information when they need the information.
- The quality of the thinking, including whether the evaluation engages in deep analysis, seeks alternative explanations, situates findings within the literature, and uses systems thinking.
- The credibility and legitimacy of the findings, including whether people are confident in the ‘truth’ being presented.
- Responsiveness to the cultural context, including the integration of stakeholders’ values and definitions of success, as well as who helps to interpret the findings.
Are you attending the annual conference? Come join us for an interactive discussion on how to reframe rigor and quality in your evaluations.