This page is part of the Clinical Guidelines (v2.0.0-ballot: STU2 Ballot 1) based on FHIR (HL7® FHIR® Standard) R4. The current version which supersedes this version is 1.0.0. For a full list of available versions, see the Directory of published versions
Within the overall framework of knowledge engineering described in the approach page, this implementation guide describes a methodology for moving between the knowledge levels. Note that this methodology emphasizes that the knowledge levels are a continuum of knowledge representation, they are not intended to be rigidly interpreted discrete categories, rather they serve as a framework for discussing successively more structured and formal representations of knowledge content to aid in the understanding, communication, and implementation of guideline content.
NOTE: The methodology documented here is a general framework for applying the approach to produce standards-based, computable guideline content. For a detailed illustration of following the approach and methodology described in this implementation guide, refer to the COVID-19 CPG Methodology Case Study.
Figure 3.1 below, FHIR-based Knowledge Representation Specifications, depicts four categories of specifications, with representative examples of each category, illustrating how the various pieces can be used together to deliver shareable clinical reasoning artifacts.
Following this model, a computable guideline is a FHIR Content Implementation Guide, and specifically, a content IG that conforms to the profiles and conformance requirements established by the CPG-on-FHIR IG.
Figure 3.1 Types of implementation guides: Specification, model, and content IGs
The following sections describe this diagram and the related specifications in more detail, and the subsequent sections of this topic summarize the conformance requirements and the proposed structuring and packaging methodology.
The foundational standards on the bottom row of the diagram include the layers of FHIR, as well as expression language and integration standards including FHIRPath, Clinical Quality Language (CQL), CDS Hooks, and SMART.
The middle row on the left of the Figure 4 shows the Model Implementation Guides (IGs), typically derived from FHIR Administration and Clinical resources such as Patient, Encounter, and MedicationRequest. These Model IGs are typically built to address a broad range of use cases, focused on a particular target realm or domain.
The middle row on the right of the Figure 4 shows the Specification Implementation Guides, which derive from the FHIR Clinical Reasoning resources to provide implementation guidance and conformance requirements for the creation, distribution, evaluation, and maintenance of shareable clinical knowledge. These include the Quality Measure IG (QM), Data Exchange for Quality Measures (DEQM), the Clinical Practice Guidelines IG (CPG-on-FHIR), and Evidence-based Medicine on FHIR (EBM-on-FHIR).
In the top row of Figure 4, the Content Implementation Guides are FHIR Implementation Guides. These IGs are not necessarily balloted as HL7 standards; rather, they use the FHIR publication toolchain to support authoring and distribution as depicted in the rest of the diagram. The content is stewarded by separate authorities such as quality agencies, specialty societies, and guideline developers; groups that have their own governance and maintenance policies. The content IGs conform to the specification IGs on the right of row 2, and typically make use of the model IGs on the left of row 2 to define content focused on a particular realm.
Overall, the methodology used by this implementation guide consists of the following steps:
Note that throughout the process of applying this methodology, the various teams will necessarily make decisions, balancing trade-offs between effort, feasibility, and value. The decisions, and especially the rationale for how they were made, are in and of themselves important products of this process, and should at the very least be captured as supplemental documentation within the content implementation guide. Often, these decision points will surface as configuration capabilities in the resulting knowledge artifacts, enabling implementation sites to understand the trade-offs, and make informed decisions about appropriate configuration. Surfacing these capabilities and decisions with appropriate documentation is critical to fully enabling reusable, shareable knowledge artifacts.
The select step involves selecting content and recommendations for implementation, as well as determining the implementation approach for the recommendation. The approach section provides detailed information on the many factors to be considered as part of this process. For the purposes of this methodology, we focus on the output of that process as having identified:
The outcome of this step is typically in the form of narrative (L1 content) requirements, sometimes accompanied by explanatory figures and diagrams (approaching L2).
For a more detailed discussion of this step, refer to the Select Step topic.
The represent step involves further refining and detailing the concepts, activities and processes identified in the select step to create semi-structured (L2) content that is domain-specific (i.e. clinician-oriented), but informed by health informatics concerns to ensure accurate and unambiguous communication of guideline intent. This is accomplished by producing:
For a more detailed discussion of this step, refer to the Represent Step topic.
The translate step involves expressing the semi-structured (L2) data elements, activities, and flow diagrams as structured (L3) content. Specifically, as FHIR profiles (i.e. case features), CPG activities, CQL logic, and CPG recommendations, strategies, and pathways.
For a more detailed discussion of this step, refer to the Translate Step topic.
The validate step involves ensuring that the structured content meets the requirements and expectations for conformance and intent, and that it behaves as expected. This typically involves the creation of at least a positive and negative scenario, or test case, that can be used to test the behavior of the structured content. In practice, this requires the ability to execute the L3 content in at least simulated environments. For content structured according to the CPG, and delivered via standard mechanisms such as SDC Questionnaires, Clinical Reasoning activities and processes, or CDS Hooks, there are reference implementations and sandbox environments that can provide this level of validation.
For a more detailed discussion of this step, refer to the Validate Step topic.
The select step involves selection of the specific content for implementation, along with the target enablement strategies, specifically identifying the who, what, when, where, and why for the selected content.
There are numerous factors to consider in selecting which recommendations and activities to develop, including:
These considerations will then inform the selection of recommendations and related activities for implementation along the continuum of Levels of Enablement. Specifically highlighting the portion of that continuum from delivery of contextualized narrative (i.e. messages only) through actionable intervention (i.e. CDS Hooks requests):
This step is typically performed by a cross-functional team consisting of at least:
In addition, technical and informatics specialists from specific (or at least representative) target implementation sites can provide valuable feedback during this step to ensure implementation feasibility.
Personas are narrative descriptions of the actors and roles, or the who, referred to by the guideline. For example, nurse midwife, community healthworker, and pregnant mother are all examples of personas.
To provide a potential starting point for formally identifying personas, this implementation guide provides a Common Personas CodeSystem based on the WHO recommendation for Classifying health workers. This recommendation uses codes from the International Standard Classification for Occupations but defines several additional categories of workers. In addition, the codes in that recommendation are focused on health workers, so codes for patient and care partner personas need to be considered as well.
Where a code from the ISCO exists, it is used. Where a WHO recommended health worker category is used, a code is constructed beginning with a W
. Where a code is introduced by this implementation guide, it is constructed beginning with a C
.
Content conforming to this implementation guide MAY identify personas using the Common Personas code system.
Activities identified as part of the select step are the narrative description of what needs to happen. This implementation guide defines the Activity Type CodeSystem to support computable representation of clinical activities:
Processes identified as part of the select step are the narrative description of when something needs to happen, particularly as it relates to clinical care delivery.
The common processes identified in this implementation guide are based on surveys of guideline content, both in the examples used in this guide, as well as other guideline content from various clinical domains and guideline authors. The result is the definition of a common pathway that is intended to be applicable in any guideline-based care content.
Using this common pathway, content can indicate where in the overall process of care delivery a given recommendation or intervention should occur.
Content conforming to this implementation guide MAY identify processes using the Common Processes code system.
Settings identified as part of the select step are the narrative description of where something needs to happen.
For settings, this implementation guide uses the HL7 V3 ServiceDeliveryLocationRoleType value set, as it contains generally applicable, broadly standardized settings for use in the computable representation of guideline recommendations.
Content conforming to this implementation guide MAY identify settings using the HL7 V3 ServiceDeliveryLocationRoleType value set.
Two aspects of supporting information are considered for recommendations. First, supporting documentation related to the guideline content itself, and second, pertinent positive or negative information related to determining the applicability of a recommendation to a particular patient.
Supporting documentation for a recommendation is critical to ensuring the definition can be traced back to reliable and credible sources. The Supporting Documentation topic in the base FHIR specification describes the basic mechanism for achieving this with the Clinical Reasoning resources.
Clinical guideline content typically assigns ratings for aspects of guidance such as the quality of evidence, or the strength of the recommendation. However, due to the broad range of use cases and rating needs, different guidelines use different rating systems and grading characteristics to communicate this information. Following the base guidance provided as part of the FHIR Clinical Reasoning module, this implementation guide provides support for the following rating concepts:
Content conforming to this implementation guide SHALL provide references to supporting documentation for a recommendation in the form of a relatedArtifact
element specifying a citation to the guideline.
Due to the variability in rating systems among guideline content, this implementation is not prescriptive about what grading system should be used, and the example bindings from the base specification apply. However, content conforming to this implementation guide SHALL identify a rating system and use that to provide rating information for recommendations and guidance, and provide documentation on what rating system was selected and how it is communicated through the content.
Content conforming to this implementation guide SHALL provide the strength of a recommendation using the strengthOfRecommendation extension.
Content conforming to this implementation guide SHALL provide the quality of evidence for a recommendation using the qualityOfEvidence extension.
Content conforming to this implementation guide MAY provide the directionality of a recommendation using the directionOfRecommendation extension.
The CPGComputablePlanDefinition profile establishes the qualityOfEvidence, strengthOfRecommendation, and directionOfRecommendation extensions as appropriate for use on PlanDefinition instances to communicate ratings.
NOTE: The CPG-on-FHIR project is seeking feedback on whether this implementation guide should nominate a specific grading system and require conforming content to provide mappings when they use other grading systems.
Documenting pertinent information related to a recommendation is critical to helping the intended recipient determine why the recommendation applies in each particular case. For definitional content, pertinent information can typically be inferred from the content referenced by the logic involved. To support cases where pertinent information cannot be inferred from the content, as well as to allow content developers to highlight particular aspects, the input
and output
elements of the PlanDefinition can be used to identify pertinent information for the recommendation.
The represent step involves taking the narrative input from the select step, the who, what, when, where, and why, and further refining it to produce a data dictionary of concepts, functional descriptions of the criteria involved in the recommendations, and flow diagrams illustrating how the recommendations should be applied in a clinical context. This step is typically performed by a cross-functional team consisting of at least:
At this step, concepts referred to by the narrative are identified and represented as data elements within a data dictionary. This content typically takes the form of a spreadsheet, or even a table within a text document, and identifies, for each data element:
For example, the following data dictionary excerpt shows the data elements for recommendation 5 of the Opioid Prescribing Guideline example included in this implementation guide:
For criteria, the represent step involves further refining the narrative from the select step by creating a functional description, or a detailed, domain-specific description of the recommendation that is effectively clinical pseudo-code. For example, given the recommendation statement for Recommendation 5 from the CDC Opioid Prescribing Guideline:
When opioids are started, providers should prescribe the lowest effective dosage. Providers should use caution when prescribing opioids at any dosage, should carefully reassess evidence of individual benefits and risks when considering increasing dosage to ≥50 morphine milligram equivalents (MME)/day, and should avoid increasing dosage to ≥90 MME/day or carefully justify a decision to titrate dosage to >90 MME/day (recommendation category: A, evidence type: 3).
The following functional description illustrates, in precise and clinical terms, how the recommendation could be applied in a clinical setting:
This functional description provides a clear representation of what should happen, when, where, to whom, and by whom, tying together all the aspects of the recommendation, and facilitating communication of those aspects to the knowledge engineers in the translate step.
There are a wide variety of methods and approaches available for representing and communicating the details of a workflow. One of the simplest, and most widely used is the flowchart, a versatile diagramming tool with virtually ubiquitous authoring and viewing support, and that is immediately understandable. For these reasons, this methodology focuses on the use of flowcharts to visually represent processes. In particular, the processes that have been most useful are simple flow diagrams that provide a visual representation of the functional description.
For example, the following flow diagram illustrates the functional description for the Opioid Prescribing Guideline recommendation 5:
Of particular importance at this step is representing the selected level of enablement for the processes and activities involved in the recommendations, as discussed in the following sections.
At this level of enablement, recommendation content is delivered as contextualized narrative, or informational messages. This is typically represented simply by the Send message activity. For example, if the intended delivery for Opioid Guideline recommendation 5 is just an alert statement, like a reminder or notification, then all that is needed is the Send message activity.
At this level of enablement, recommendation content is delivered as a structured activity that can be understood and acted upon by the consuming system. For example, the Opioid Guideline recommendation 5 at this level of enablement might be an updated order with a reduced dosage, or a recommendation to order an alternative non-opioid therapy. This level of enablement can use any of the available activity types, but only specifies integration at the request level. In other words, the result of the decision support is a proposal for a recommended activity, but whether and how that proposal is accepted and fulfilled is not specified by the computable content, rather implementation sites determine how a request becomes an event.
Note that even in the case that the decision support is only concerned with delivering actionable interventions, computable content can still communicate expectations for what a fulfilled request (or event) looks like, which still allows outcome and process metrics to be expressed.
At this level of enablement, recommendation content is delivered not only as structured activities using the request resources, but the content also describes expected fulfillment workflow to get from the request to the actual event. For example, the content for recommendation 5 might include the process not only for prescribing the alternative non-opioid therapy, but for fulfilling that request by actually filling the prescription. Specifying recommendations to this level of enablement is significantly more involved because of the variability of fulfillment workflows across sites.
The translate step involves using the narrative (L1) and semi-structured (L2) outputs from the select and represent steps to create formal, computable, standards-based representations of guideline content. This step is typically performed by a team consisting of at least:
A complete walkthrough of this process using freely available open source tools is available at the Content IG Walkthrough.
As discussed in the content implementation guides section, a computable Clinical Practice Guideline (CPG) following this methodology is delivered as a FHIR Implementation Guide, and the first step to developing the content is to establish the content IG. This typically takes the form of a Github repository, similar to the Sample Content IG.
Pay particular attention at this step to the selection of the canonical URL for the computable guideline, as it will determine the base of the URLs for all the artifacts published by the IG, as well as enable resolution of artifacts when the IG is published at its canonical URL.
Many FHIR implementation guides are Health Level 7 (HL7) balloted specifications, and as such are published on the HL7 FHIR website. For example, the US Core implementation guide is published at http://hl7.org/fhir/us/core. However, for content IGs, the steward of the content will typically be the publisher, so the canonical base URL for the IG should be selected using a domain that the steward controls.
For open-source content with a commitment to maintain it, the fhir.org site can provide a publication alternative by following the FHIR Community Process.
The data dictionary and related documentation are used to produce FHIR profiles. Note that not every data element will result in a FHIR profile, the number of profiles actually used depends on a number of factors, such as the variability between data elements, as well as the anticipated need for validation, versus reference.
For example, consider two data elements, both describing a laboratory result, but differing only in the value set used to distinguish the data element. If the elements are only referenced in criteria, Observation instances can be distinguished by referring to the value sets, and both data elements can use the same Laboratory Result profile. However, if there is a need to validate the instances as data elements, specific profiles should be used.
Broadly though, data elements defined in the data dictionary result in case features, or profiles that use the CPGCaseFeatureDefinition profile. Case features fall into two categories:
Note that an inferred case feature may be recorded as part of the patient record, so that it effectively becomes an asserted case feature. And it may also be the case that asserted case features can also be calculated based on other values in the patient record.
Because case features are aspects of the patient’s record (or typically events that have occurred, as opposed to are planned), case features will typically be event resources.
Content conforming to this implementation guide SHOULD use the CPG Event Activity Profiles as the basis for case feature definitions.
In addition, content implementation guides will typically select a target realm for deployment, and SHOULD select a set of Interoperability Profiles appropriate for that realm.
For terminology, the translate step involves identifying and/or constructing appropriate value sets for the concepts referred to in the selected recommendations. This step is typically performed by the terminologist with help from the knowledge engineering role to assist in creating FHIR representations. For terminology servers that already support authoring and distribution of FHIR ValueSet resources, this is less of an issue, but often there are not complete solutions available, and additional tooling is required to support the terminology publishing needs.
For more discussion on the creation and distribution of value sets, see the Terminology page.
For criteria, the translate step involves expressing the inclusion and exclusion criteria, as well as any related logic, using Clinical Quality Language (CQL). This step is typically performed by a knowledge author using authoring environments such as CDS Connect or the Atom CQL Language Plugin.
Once the CQL libraries are authored, they can be included in the content implementation guide using the binary adjunct loader feature of the FHIR IG publisher.
For more discussion on the creation and distribution of libraries, see the Libraries
For activities, the translate step involves expressing the activities to be performed as part of the recommendations. Across guideline-based care, there are many different types of recommendations that may be given, such as the recommendation to conduct a specific test, to prescribe a specific medication, recommendation for a particular procedure, and many others. Following the general workflow pattern established in FHIR, the activities within these recommendations are represented by the following patterns:
Definitional activities are represented using the ActivityDefinition resource, which acts as a template to describe the patient-independent aspects of the activity, and provides a means to build expressions to dynamically determine the values for patient-dependent aspects of the activity as part of applying the definition to a particular patient context.
Readers of this implementation guide should refer to the Definitional Artifacts topic in the FHIR specification for general information on how ActivityDefinition resources are used to describe the activities that can be performed.
When applying ActivityDefinition resources in a patient context, the following specific parameters are available:
Name | Description |
---|---|
Patient | The current Patient in context |
Practitioner | The current Practitioner interacting with the clinical system |
Encounter | The Encounter in progress, if one is currently in progress |
For convenience, this IG provides pre-built parameterizable ActivityDefinition instances for each of the activity types:
Activity | Parameters |
---|---|
AdministerMedication | MedicationRequest: The detailed medication request to be administered. This may be an existing order, or it may be produced as part of the administer medication activity |
CollectInformation | QuestionnaireCanonical: The canonical URL of the questionnaire to be used to collect the information |
CommunicationRequest | CategoryCodeableConcept: The category of communication |
DispenseMedication | MedicationRequest: The detailed medication request to be dispensed. This may be an existing order, or it may be produced as part of the dispense medication activity |
DocumentMedication | MedicationRequest: The detailed medication request to be documented. This may be an existing order, or it may be produced as part of the document medication activity |
[Enrollment] | PathwayCanonical: The canonical URL of the pathway or strategy in which the patient is enrolled/unenrolled |
[GenerateReport] | DefinitionCanonical: The canonical URL of the metric, case report, or profile to use to generate the report |
ImmunizationRequest | VaccineCodeableConcept: The vaccine being recommended |
MedicationRequest | MedicationCodeableConcept: The medication being proposed, as a CodeableConcept DoseQuantity: The quantity of medication DosesPerDay: The number of doses per day, as a decimal |
ProposeDiagnosisTask | DiagnosisCodeableConept: The diagnosis being proposed |
RecordDetectedIssueTask | IssueCodeableConcept: The issue being recorded |
RecordInferenceTask | InferenceCodeableConcept: The type of inference being recorded InferenceValue: The value of the inference being recorded |
ReportFlagTask | IssueCodeableConcept: The issue being flagged |
ServiceRequest | ServiceCodeableConcept: The type of service being proposed |
Proposed activities are represented with request resources, and for each type of activity, this implementation guide defines a proposal profile. These are effectively the recommended action to be taken as part of a guideline recommendation, and in general they each specify:
This implementation guide defines profiles for each of the request resources to be used as recommendation instances:
These profiles are intended to be used for proposals, and so are modeled with the proposal intent. Note that some implementations may go further and use the plan
or even order
intents of a request, depending on the capabilities of the systems involved.
Content conforming to this implementation guide SHALL use one of the recommendation instance profiles described above to ensure supporting information related to the recommendation is preserved through the recommendation instance.
Specific guideline content MAY define derived profiles for recommendation instances establishing additional constraints.
And finally, for each of the activity types, this implementation guide defines an event profile for modeling when the activities have actually occurred. Each event profile typically describes:
instantiatesCanonical
element, referring back to the guideline (definitional) contentbasedOn
element, referring back to the request the event is fulfillingNote that because event resources are part of the patient’s case, it is often the case that activity events are also case features, so each of these profiles also provides support for specifying:
The following table details the event profiles for each activity type. Note that some activities have multiple event profiles.
Activity | Event |
---|---|
Send a message | CPGCommunication |
Administer medication | CPGMedicationAdministration |
Collect information | CPGQuestionnaireResponse |
Dispense medication | CPGMedicationDispense |
Document medication | CPGMedicationStatement |
Enrollment | CPGCase |
Generate report | CPGMetricReport CPGCaseSummary CPGCasePlanSummary CPGCasePlanProgressingNote |
Order a medication | CPGMedicationDispense CPGMedicationAdministration CPGMedicationStatement |
Recommend an immunization | CPGImmunization |
Order a service | CPGProcedure CPGObservation |
Propose a diagnosis | CPGCondition |
Record a detected issue | CPGDetectedIssue |
Record an inference | CPGObservation |
Report a flag | CPGFlag |
Content conforming to this implementation guide SHALL use the event profiles defined above to ensure guideline supporting and tracking information is available for activities delivered in the context of a computable guideline.
This implementation guide uses the PlanDefinition resource to represent the recommendations of a guideline as an event-condition-action rule. The recommendation used in this way specifies:
Readers of this implementation guide should refer to the Applying a PlanDefinition topic for details on how an event-condition-action rule can be applied to a particular patient to produce guidance appropriate to that patient.
Content conforming to this implementation guide SHALL use the cpg-recommendationdefinition profile to represent recommendation definitions.
This implementation guide uses the PlanDefinition resource to represent specific sequences of activities executed at a point-in-time as a strategy. Readers of this implementation guide should be familiar with the Workflow topic in the base FHIR specification. This implementation builds on the guidance there, providing some specific patterns for describing common activities as part of strategies. Specifically:
Within these strategies, recommendations for guideline-based care are attached via triggers at the appropriate point in the workflow. In other words, rather than having a strategy directly refer to a particular recommendation, the recommendation logic is attached to the event and triggered at the appropriate point. This event-driven approach allows implementations to separate workflow processing from guidance evaluation.
Content conforming to this implementation guide SHALL use the cpg-strategydefinition profile to represent strategy definitions.
This implementation guide uses the PlanDefinition resource to represent the overall activities involved in a guideline as a pathway. For the purposes of this implementation guide, a pathway differs from a strategy in that a pathway describes a general set of activities to be applied over time, typically with multiple time-based or event-based entry points across the pathway.
For example, the WHO antenatal care guideline recommends an overall contact schedule consisting of eight contacts at specific points during the pregnancy. This is represented using a pathway with actions for each of the contacts, where each action defines an applicability criteria that identifies the recommended timing of the encounter.
The activities in a pathway used in this way are references to other strategies or recommendations, rather than particular activities to be performed.
The choice between representation using recommendations, strategies, and pathways depends on a variety of factors, largely driven by:
We use the term pathway in this methodology for a more flexible representation of the guidance, as opposed to a protocol which would typically be a more rigid description of a particular process.
Also note that depending on the guideline, there may be aspects of the pathway that are expected to be more rigid (i.e. closer to protocols), depending on the clinical specification.
Content conforming to this implementation guide SHALL use the cpg-pathwaydefinition profile to represent pathway definitions.
The validate step involves ensuring that the structured (L3) content produced matches the intent of the (L1) and (L2) content for the guideline. This implementation guide provides several approaches for validation, including conformance requirements, L2, L3, and L4 checklists, as well as supporting several layers of testing.
The Conformance topic summarizes the conformance requirements established by this implementation guide.
To help facilitate the process of communication of requirements between collaborators, and to provide a measure of completeness with respect to artifacts used in and developed as part of this methodology, this implementation guide provides three checklists, associated with the semi-structured, structured, and executable knowledge representation levels. These checklists are intended as a tool to help identify requirements and ensure completeness of artifacts:
Checklist | Description |
---|---|
L1 Checklist | Used as a tool to help guideline developers develop L1 in clearer, consistent, and more structured approach |
L2 Checklist | Used to ensure that informaticists and knowledge engineers have what they need from L1 content in order to successfully produce L3 content |
L3 Checklist | Used to ensure that L3 content meets the requirements to ensure that implementers have what they need in order to successfully implement computable content in a specific setting |
L4 Checklist | Used by implementers to help support the process of implementing in a specific setting, helping to ensure successful and faithful implementation of guideline content |
The agile methodology proposed by the approach section emphasizes the importance of regular review of content as part of the computable content development process. However, it is often the case that guideline development and derivate asset development are done by separate and fairly isolated teams. When the original guideline developers cannot be used for expert review, it is critical to ensure review by a qualified domain expert familiar with the domain the guideline covers.
In addition, engagement from target implementation sites is key, and the best-case scenario is to have site engagement as part of the development process. Again, this is often not the case, so seeking implementation review from at least one, if not several target implementation sites can provide valuable feedback.
To support validation, testing scenarios should be devised based on the personas, concepts, activities, and processes defined in the narrative (L1) and semi-structured (L2) content. Ideally, these scenarios would be built out as FHIR resources, using the ExampleScenario resource as a way to completely and formally describe each scenario.
Ideally, each artifact should have at least one positive and one negative test case. In addition, edge cases and boundary conditions should be considered and where appropriate, additional test cases provided. The goal of unit testing is to ensure that each of the artifacts functions as expected in isolation.
As of the time of this writing, unit testing for Libraries can be accomplished using the Atom CQL plugin by creating test folders for each test case. See the Adding Test Cases topic in the Content IG Walkthrough for more information.
Component testing involves ensuring that the components of a system work as expected. Ideally, components are tested with as few dependencies as possible. For L3 decision support content conforming to this implementation guide, that content can be tested with freely available, open-source reference implementations. In particular, the CQF Ruler reference implementation is capable of evaluating L3 decision support content, and exposing that evaluation capability as a CDS Hooks service.
See the Quick Start in the Opioid Prescribing Guideline for an example of how this type of testing can be done.
Integration testing involves ensuring that systems work together as expected. For example, once the L3 content for a guideline is loaded in a CQF Ruler, it can be tested with the CDS Hooks Sandbox, an open source sandbox for testing CDS Hooks Services that simulates a clinical system acting as a CDS Hooks client.
See the Validation with CQF Ruler and CDS Hooks topic of the Content IG walkthrough for an example of how to perform this type of testing.
The Packaging topic describes how content, including test cases, are packaged for distribution.
The primary methodology discussed in this implementation guide uses FHIR resources to represent both patient information and the computable content of a guideline. There are other approaches, and the Mapping to BPM+ topic discusses how the FHIR representations can be bi-directionally mapped to and from BPM+ content.