Guideline Development Frameworks

The following frameworks are used in guideline development and across the evidence ecosystem:

Knowledge Synthesis and the Evidence Pyramid

Knowledge synthesis is the process in the professional community of practice that applies and develops methodologies for systematically evaluating a corpus of evidence related to a particular condition or intervention. This includes deciding what evidence from across the evidence ecosystem to evaluate¹ as well as various methods for performing this evaluation². One factor that is taken into consideration is the validity and/or quality of evidence. In evaluating the validity of evidence, one factor that is taken into consideration is the Evidence Pyramid, discussed below. The quality of evidence may be systematically evaluated, often through knowledge synthesis methodologies such as meta-analyses, systematic reviews, evidence summaries, or rapid reviews, and applying frameworks such as GRADE, discussed further below.

Meta-Analyses and Systematic Reviews (Syntheses)

A meta-analysis is a statistical analysis that combines the results of multiple quantitative studies (e.g. Randomized Controlled Trials (RCT’s), cohort studies) that address the same question, with each individual study reporting individual study results that are expected to have some degree of error. The objective is to use statistical approaches to derive a pooled estimate closest to the ‘unknown truth’. Meta-analyses yield a weighted average of the results of the individual studies and have the capacity to contrast the results from different studies, identify patterns among study results or potential sources of disagreement among those results, or other relationships that become evident in the context of multiple studies. This approach of aggregating information leads to a higher statistical power and more robust point estimate than is possible from any individual study. (ref)

A systematic review is a summary of the literature that uses a systematic approach to identify and critically appraise publications or evidence from the literature on a given topic. Systematic reviews are intended to provide a comprehensive and exhaustive summary of current evidence that is ”methodical, comprehensive, transparent, and replicable.” (ref) systematic reviews may relate to interventions (benefits and harms), use of diagnostic testing to detect a particular disease, prognosis or probable course for future outcomes, risks of developing a given condition, or even a systematic review of systematic reviews to consolidate holistic evidence about a given topic (e.g. disease, procedure).

FIG. 21. The Evidence Pyramid refers to a means to evaluate the quality or validity of evidence (higher toward the top) as well as provide a visual depiction of the amount of evidence (width)[^3]. The evaluation of the evidence not only informs the guideline development process and resulting recommendations but is further communicated together with the recommendations themselves using the GRADE framework, described below.

PICOTS Typology - Patient population, Intervention, Comparator, Outcomes, Timing, Setting. The PICOTS Typology may be used for scoping the guideline and individual recommendations and the analysis of the supporting evidence or studies (e.g., meta-analyses, systematic reviews, and the individual studies that serve as their body of evidence). There are implications to guideline eligibility, applicability criteria for individual recommendations (the “condition” portion of ECA rules), and various data element and terminology semantics across the CPG.⁴;⁵ For more on using PICOTS within HL7 Standards, see the EBM-on-FHIR initiative.⁶

GRADE (Grades of Recommendation Assessment, Development and Evaluation) GRADE describes a process and a structured framework for conveying information to develop a common, transparent, and sensible system for grading the quality of evidence and the strength of recommendations. Quality of evidence and strength of recommendations have implications for not only recommendation metadata and implementation approaches. GRADE distinctly separates the evaluation of the quality of evidence from the evaluation of strength of recommendations.⁷;⁸;⁹;¹⁰;¹¹

---

1: https://cebgrade.mcmaster.ca/guidelinechecklistonline.html#Deciding

2: https://cebgrade.mcmaster.ca/guidelinechecklistonline.html#Summarizingtable

3: http://dx.doi.org/10.1136/ebmed-2016-110401

4: https://cebgrade.mcmaster.ca/guidelinechecklistonline.html#PICOtable

5: https://www.ncbi.nlm.nih.gov/books/NBK98234/table/ch

6: http://hl7.org/fhir/clinicalreasoning-evidence-and-statistics.html

7: https://cebgrade.mcmaster.ca/guidelinechecklistonline.html#Judgingtabl

8: https://www.ncbi.nlm.nih.gov/pubmed/23312392

9: https://www.ncbi.nlm.nih.gov/pubmed/21208779

10: https://www.cdc.gov/vaccines/acip/recs/grade/downloads/guide-dev-grade.pdf

11: https://gdt.gradepro.org/app/handbook/handbook.html