Background: The ability to interpret results of patient-reported outcome measures (PROMs) and to judge the magnitude of effect (for instance, as large, moderate, small but still important, or negligible) is critical for their use in healthcare decision making. The most common reference point for PROMs’ interpretation is the minimal important difference (MID), which provides a measure of the smallest change in a PROM that patients experience as important. Development of MIDs is ideally done by relating their results to an anchor measuring a similar construct that is itself interpretable. Guidance on determining the credibility of MID estimates generated in this way remains to be developed.

Objective: To develop a core instrument to measure the credibility of MID estimates from all relevant studies, and an extension with items that assess credibility when studies use a transition instrument as an anchor.

Methods: To inform the development of a new instrument addressing the credibility of empirically ascertained MIDs, we conducted a systematic survey summarising and appraising available methods to generate anchor-based MIDs. Iterative discussion among the team led to the development of the core credibility instrument. In a case study, we applied the core criteria and found the items insufficiently discriminatory when the anchor was a transition rating, and thus developed additional items for this context.

Results: The core instrument includes the following items: the anchor represents a patient rating and is interpretable to the patient and clinician; the precision around the estimate; and the correlation between the anchor and PROM. The extension includes the following items: the authors select a threshold on the anchor that reflects a small but important difference, the time elapsed between baseline and follow-up measurement for MID estimation is optimal, and correlation of the transition rating with the pre, post and change score in the PROM.

Discussion: Our new instrument will allow users to determine the extent to which the design and conduct of studies measuring MIDs are likely to have protected against misleading estimates.