Background: Performance measures can have a greater impact than guidelines or evidence syntheses for altering behaviour and decision making in healthcare services. Despite extensive development of critical-appraisal criteria for assessment of evidence of different types, and for assessment of guideline-development methodology, there is little formal development of criteria to assess the quality or appropriateness of performance measures.

Objectives: We developed criteria to evaluate the appropriateness of performance measures.

Methods: We evaluated the reviews of performance measures conducted by the Performance Measurement Committee of the American College of Physicians to identify reasons for not supporting specific performance measures as appropriate for implementation. We adapted experience in applying critical-appraisal criteria on scale for evidence (tens of thousands of instances per year) and recommendations (thousands of instances per year) to develop reproducible criteria for appraisal of performance measures. We appraised performance measures in two domains (diabetes and heart failure) with iterative revisions to the appraisal criteria to develop a stable set of appropriateness criteria. We are validating use of these criteria across the core set of performance measures used in the United States.

Results: The appropriateness criteria for a process measure (a measure of implementation of a process of care as a quality indicator) are:
1. Convincing evidence that action changes clinical outcomes.
2. Desirable consequences outweigh undesirable consequences (including consequences of performance measure implementation).
3. Population adequately specified with appropriate exclusion criteria.
4. Intervention adequately specified including appropriate intervals or frequency.

Modified criteria are available for outcome measures and for inverse process measures (a measure of the absence of implementation of a process of care).

Conclusions: Explicit criteria for appropriateness of performance measures can be defined and applied. Such criteria can be used to identify measures with the greatest potential to provide most benefit.