Background: Real-world evidence (RWE) can be defined as information obtained from heterogeneous sources outside clinical or academic research settings, and as a complement to data from experimental studies. We explored how RWE is produced, transformed, and, finally, used to inform decision making at different levels of the healthcare system.

Objectives: Our aim was to collect and employ RWE to develop and refine a framework mapping the suite of decision-making processes in delivery of primary healthcare, and how data and tools from Health Information Systems (HIS) are used to inform these decisions.

Methods: In the context of a research project focused on improving HIS in three African countries, we carried out a systematic review on the effects of interventions to improve HIS. After synthesising data from experimental studies, we hypothesised that the evidence synthesis should be complemented by local evidence. We therefore stepped into the 'real world' to gain insight on the utilisation of HIS for decision making in the daily practice of frontline health workers, district managers, policy makers and other stakeholders by using interviews and direct observation techniques. Armed with this RWE, we constructed a health information-based decision framework. We then re-defined this framework with RWE from observational and qualitative studies obtained via our initial literature search in a framework synthesis.

Results: Information obtained from stakeholders and observations revealed that the current HIS is largely donor driven, and thus focused on meeting downstream data collection, rather than the data needs of healthcare workers for decisions in their daily practice. These findings forced us to adapt our theoretical framework twice, leading to the definition of essential ‘functions’ of the HIS for clinical, managerial and public health decision making (see Figure).

Conclusions: Collecting and employing 'real-real world evidence' (RRWE) supports researchers in maximising the relevance of global evidence syntheses to real-world situations and needs. RRWE should be routinely considered, especially in syntheses involving complex health systems.