R6 Ballot (2nd Draft)

Publish-box (todo)

FHIR Infrastructure icon Work GroupMaturity Level: 0 (Draft)Standards Status: Trial Use

The FHIR Specification includes a mapping language. The mapping language has a concrete syntax, defined and described in this page, and an abstract syntax, which is found in the StructureMap resource. See also the Tutorial.

The mapping language describes how one set of Directed Acyclic Graphs (an instance) is transformed to another set of directed acyclic graphs. It is not necessary for the instances to have formal declarations and/or be strongly typed - just that they have named children that themselves have properties. On the other hand, when the instances are strongly typed - specifically, when they have formal definitions that are represented as Structure Definitions, the mapping language can use additional type related features.

Note that while the mapping language applies to directed acyclic graphs that allow for poly-hierarchies, most of the contexts of use are strict mono-hierarchies (a.k.a trees), including the types defined as part of the FHIR specification.

The mapping language addresses two very different kinds of transformations:

  • Structural changes between the source and target structures
  • Differences in content and formats in string (and related) primitives contained within the structures

A map has 7 parts:

  • Metadata
  • Embedded ConceptMaps to translate between different code systems
  • References to the structures involved in the mapping
  • Imports: additional Maps used by this map
  • Constants: constants used in the mapping rules
  • A series of groups, each with a list of input variables
  • A series of transformation rules in each group

The Mapping Language and the StructureMap resource are built on top of FHIRPath, and a FHIRPath implementation is required in order to execute a StructureMap.

Maps are executed by a mapping engine. This takes one or more inputs of instances (directed acyclic graphs) and a map, and produces a set of outputs as specified by the map. The exact details of the form that the instances take are a matter for the map engine / application API. This language assumes that the engine can query an element in the instance for its children, its primitive value, and (optionally) its type. The language also assumes that the engine has application support for the following operations:

  • ValueSet validation operation
  • Translation operation
  • Lookup another tree of data
  • Create an instance tree
  • Return the correct string format to refer to a tree (input or output)

These functions constitute a Mapping Support API that makes maps portable between different systems

Generally, it is assumed the invocation of the engine follows some pattern like this:

  • The host application creates the engine, and passes it a handle to the standard services
  • The host application chooses the structure map resource, and asks the engine to prepare it (load, check, cache-up etc.)
  • The host application asks the engine to execute and provides a set of resources that match defined inputs in the map
  • Any created output will be created via the standard API

Some host applications may be able to determine how to combine maps and inputs on the fly based on their metadata, and require minimal configuration, while others may require manual arrangements in order to manage the map execution process.

Mapping files are always plain text in Unicode. Whitespace is any Unicode whitespace, and the particular whitespace used is not significant, except that Unicode end of line characters terminate a comment. Comments are started by the characters "//" and can be found anywhere.

The abstract model includes documentation for each item. The canonical text representation is for each item to be on its own line, with documentation at the end of the line as a comment.

All names defined by the map language - group, rule and variable names - must be valid ids (1-64 characters, upper and lowercase letters, numbers, dashes and dots). To avoid parsing ambiguities however, they SHALL start with a character, cannot be one of the keywords used in the language (see section Reserved Keywords below) and cannot contain a dot or dash. These id values SHALL NOT be surrounded with backticks. The names of context elements (source.context and target.context) are not constrained to simple names in the same way since they come from the source and target data models, and these may be surrounded with backticks.

   src  document4  `not-found`  `section4.5`  `group` 

The media type for the FHIR Mapping language is text/fhir-mapping. Note that in order to conform to RFC 6657 icon, the charset=utf-8 parameter SHALL be appended.

The first part of a mapping file defines the metadata about the map using the metadata from the StructureMap resource. Metadata is defined using the syntax /// name{.property}* = {value} where:

  • The name is the same as the element name in the StructureMap
  • The property allows for sub-elements to be populated. The property can repeat
  • Additional items with the same name represent repeats
  • The value is a literal representation of the primitive value in FHIRPath format. Extensions are not supported
  • The value for a complex element is empty
  • There is a special representation for markdown: the value is started and finished with """ and may cover multiple lines of markdown

Examples:

/// url = 'http://hl7.org/fhir/StructureMap/CodeSystem3to4'
/// name = 'CodeSystemR3R4'
/// title = 'R3 to R4 Conversions for CodeSystem'
/// experimental = true
/// status = 'active'
/// jurisdiction =
/// jurisdiction.coding = 
/// jurisdiction.coding.system = 'urn:iso:std:iso:3166'
/// jurisdiction.coding.code = 'AQ'

/// description = """
You should use my *awesome* map.
It does really cool things.
"""  

Values for url and name SHALL be provided. If no value is provided for status, then the status "draft" is assumed.

The next optional section of the map references the set of structure definitions that are used or produced by this map. For example:

  uses "http://hl7.org/fhir/3.0/StructureDefinition/CodeSystem" alias CodeSystemR3 as source   // documentation
  uses "http://hl7.org/fhir/StructureDefinition/CodeSystem" as target  // documentation

This section lists one or more structure definitions that the map makes use of, and indicates for each structure definition, how it is used. It may also provide an alias - a name used for the type inside the mapping language - this may be necessary when transforming from source to target where both source and target use overlapping type names (not unusual). If no alias is given, the name for the type will default to the name given in the StructureDefinition (StructureDefinition.name).

Any kind of structure definition may be referenced, including datatypes, resources, constraints on those, and logical models.

There are 4 modes in which a structure definition may be used:

  • source: One of more instances of this type are passed to the mapping engine when the mapping is executed, and serve as the source from which mapping is performed
  • queried: The map may ask the (via the API) for some instances of this type. For further discussion, see below
  • target: One or more instances of this type are passed in, and will be populated from the source material
  • produced: The map may ask (via the API) for some instances of this type to be created. For further discussion, see below

The simplest case, which is common, is where a single structure is converted to another single structure. in this case, the map specifies one target, and one source. Such maps are easy to use automatically - the host application has content in one format, creates an empty instance of the target, and asks the mapping engine to convert.

However, many mappings are not so simple. For instance, converting from a single CDA document to FHIR typically creates a set of resources. In this case, there is a single target - a Bundle, but it is also useful to specify a set of other structure definitions for resources that may be created as part of the bundle. Alternatively, converting from one source model to another might involve looking up other information in other instances of data.

It's also possible for a map not to specify any structure definition dependencies. A map that doesn't indicate any structure definitions can still be used, but the type features of the map language can't be used, and such maps typically require special development to integrate the execution of the map into an application.

This next optional section references additional maps that are used by this map. For example:

  imports "http://hl7.org/fhir/StructureMap/*3to4"  // documentation

Maps can be broken up into several files, each containing a coherent set of groups. For example, when writing mappings for CDA to FHIR, one might have one file to map the main document, and another file containing the mappings for the datatypes (e.g. CD to CodeableConcept). How imported maps are actually used is discussed below.

The url in the import statement may contain a "*" as a wildcard character (as shown above) to include any matching maps that are available to the mapping engine.

The map may define any number of constant values that may be re-used through out the rules that follow. Each constant has the following grammar:

  let [name] = [fhirpath expression];  

This defines a reusable variable that has the same value across all the maps. The value of the variable is a FHIRPath expression terminated by the separator ';' (for syntactical simplicity, and so non-FHIRPath aware parsers can parse the syntax). Some rules that apply to the constant declarations:

  • The names must be unique within a single mapping source file
  • The names are available as source variables and can be referred to from the following elements of the mapping language: source context, and the FHIRPath expressions for where and check conditions, log statements and target expressions
  • In the FHIRPath expressions, the constants are referred to by name. This means that the constants cannot have the some name as a reserved word in FHIRPath icon or the mapping language
  • Constant expressions can be - and often are - simple literal values e.g. let name = 'example';
  • Constants can refer to other constants in their expressions, though circular expressions will result in an error
  • Constants are only evaluated when a reference to them is encountered (lazy evaluation). Once evaluated, they are not re-evaluated again with the context of the script that contains them
  • If a name of a variable in a local context is the same as a constant expression, then the local variable hides the Constant declaration

Each map contains one or more groups, each optionally containing one or more mapping rules. Each group declares a set of input and output variables that are shared by the rules. The in- and output variables define exactly which instances are passed to the mapping, and provides names by which they may be passed when invoking the map:

group [group-name] (inputs) (extends [other-group]) (<<stereotype>>) { // documentation
  .. rules ..
}  
For example:
  group CodeSystem(source src : CodeSystemR3, target tgt : CodeSystem) extends DomainResource <<type+>>
  {
    // documentation
    .. rules ..
  }

Each group has a name, which is how the mapping is invoked. The first group is special, in that this is the group invoked if no name is provided (e.g. starting the mapping by a host application).

The inputs of a group are also referred to (below) as its input parameters, or just as parameters; or as input variables, and are a comma-separated list where each items has the format:

  [mode] [name]( : [type])

Each input to the group has a name. All input variables have a mode, which may be one of source or target (see above). Inputs may have a type, but are not required to. There must be at least two input variables (source and target) - else there's nothing to map, except for the special case of the first group that may only have a single input. Groups may have additional input or output inputs, where that's necessary.

Groups may extend other groups, which means that the rules in the other group also apply (typically, this is used with specializing classes in an OO context). When a group extends another group, it SHALL have the same input parameters (by mode, name, and type if specified) though their order may differ, and it MAY have additional parameters.

The stereotypes <<types>> or <<type+>> can be added to the end of the group declaration to indicate that this group provides a set of mappings that are intended to be used as the default way to map from source to target. For more information, see the section on "Default mapping groups" below.

Default mapping groups SHALL have two parameters, a source, and a target, in that order, and both SHALL have specified types for the inputs.

The main portion of a map consists of a set of transform rules that describe how source content is transformed into target content. The full format for a rule looks like this:

src_context.field as new_variable where (condition) -> tgt_context.field = create([type]) as new_variable then [details] "name";
For example:
src.value : code as vs0 -> tgt.value = create("code") as vt0 then code(vs0, vt0) "valueCode";

Each rule has three main parts:

  • Source Content: The initial part of the rule before the '->'. Specifies one or more elements from the source that contribute to the mapping
  • Target Transform: Starts after the '->' and consists of zero or more specifications of how to use the source content to create the target content
  • Dependent Rules: Optional part starting with 'then'. Specifies one or more groups by name (separated by comma) or a set of child rules to apply within the scope of the rule

Each rule may be assigned a name, though this is usually inferred by the parser and not specified directly. The name is used in trace logs (a record generated by the conversion engine recording the transform process). Names must be unique within the context of the map. Typically, the name is trivial and can be safely and usefully generated by the engine processing the map, so this is often left out.

The three main parts are described in more detail in the following sections.

Rules may be applied in any order. In other words, the order is implementation-defined. Mappings should take care to ensure that either:

  • they do not have any overlapping rules, or
  • if they do have overlapping rules, that the order in which those rules are applied is not important, or
  • they use first and last keywords to help manage order

The source content is formed by one or more source statements, which can be assigned a variable name and then be used when specifying target content, or re-used in subsequent transforms and dependent rules. Multiple source statement are separated by a comma:

 [source], [source] -> ...

Each [source] contains the following items:

context.element {: type} {min..max} {default ([value])} { list-option } as variable where ([FHIRPath]) check ([FHIRPath])

For example:

  src.value : integer 0..* default (10) first as vs0
    where (value >= 10) check (value <= 100)
    log (value)
  • context.element: The fist part of the source statement consists of two parts: the context and the element.

    The context is an identifier which is either declared as a source for the map or as a source parameter or any named variable within the group in which this rule is nested.

    The element is an (optional) name of a child element of the context. If the name is not provided, the source is the context. If it is provided, the rule will apply once for each element on the context that matches this name.

  • type: A type may be specified. If a type is specified, only elements that have the specified type are selected as part of the source
  • min..max: Specified cardinality (may use '*' for the upper bound). If a cardinality is specified and the cardinality of the element in the source is not within the specified cardinality, the mapping engine raises an error instead of completing the transformation.
  • default: A default value, used if there is no source value found. If there's a default value on an item that can repeat, it will only be used once. Default values only apply to primitive types, and the value is a FHIRPath statement (usually a literal)
  • list-option: by default, the rule will apply once for each occurrence of the element in the source. The list option can override this to specify different behavior. Valid values are 'first', 'last', 'not_first', 'not_last' and 'only_one'
  • as variable: if an element is specified, a variable must be assigned. This variable name may be used in the target statement, and may be re-used in other dependent rules (see below). There is one special case described below (simple form) where the variable name is provided explicitly
  • where ([condition]): a FHIRPath expression that is evaluated to boolean on the context. If the expression returns false, the source element has no match
  • check ([condition]): a FHIRPath expression that is evaluated to boolean on the context. If the expression returns false, the mapping engine raises an error. Note: it is usual to have either a where or a check clause, but possible to have both. If both are provided, the where clause is evaluated first, and the check only applies if the where expression is true

If there are multiple source statements, the rule applies for the permutation of the source elements from each source statement. E.g. if there are 2 source statements, each with 2 matching elements, the rule applies 4 times, one for each combination. Typically, if there is more than one source statement, only one of the elements would repeat. If any of the source data elements have no value, then the rule never applies; only existing permutations are executed: for multiple source statements, all of them need to match.

Example

Given this mapping statement:

  for src.row as row, row.firstName as firstName 

applied to this source:

  "row" : [{
    "firstName" : "John"
  },{
    "firstName" : "Peter"
  }]

Will result in the the rule firing 4 times: row #1 & "John", row #2 & "Peter", row #2 & "John", and row #1 & "Peter". This somewhat contrived example shows how the iteration works; it's unlikely that this combination of source statements would be useful. The same mapping statement with this data:

  "row" : [{
    "family" : "John"
  },{
    "family" : "Peter"
  }]

Because there's no firstName values, the rule will never fire.

Once the source content is evaluated, the engine performing the evaluation has a list of elements assigned to variables. For each time the rule is applied, each of the variables contains a single value. These variables are now mapped into the target structures in the target transformation.

To avoid potential parsing ambiguities, all embedded FHIRPath expressions within the mapping language (where conditions, check conditions, log statements, default values) MUST be surrounded by a set of parentheses: "([FHIRPath])".

Each source can include a log statement:

 log ([expression])

Where expression is a FHIRPath statement. e.g.

 log ('not handled yet')

Puts a plain string in the log file. Alternatively, the log statement can contain FHIRPath:

 log (src.field)

Log statements are often used to note that some particular source element is not yet mapped.

Note that the log statement is equivalent to the trace() function in FHIRPath icon and the output usually ends up in the same place.

Each rule specifies zero or more target transformation statements, which specify how source content is used to create target content. These target statements can also be assigned to variables that can be used in subsequent transform rules. If no targets are specified (no ->), no transformation is done and there are no created targets, just newly defined source variables, which can then be used in subsequent dependent rules. Multiple target statements are separated by a comma, like this:

... -> [target], [target] then...

Each [target] contains the following items:

  context.element = transform_code(parameters...) as variable {list_modes}

For example:

  context.element = copy(parameter, ...) as vt1 first
  • context.element(.subElement): The first part of the target statement consists of two parts: the context and the element.

    The context is an identifier which is either declared as a target for the map, a target parameter or any named variable within the group (including the variables from the source content) in which this rule is nested.

    The element is the name of a child element that is valid in the context. The created value will be placed into the named element

    The element may be further qualified by a subElement (recursively to any further depth). src.c.p as p ... is semantically equivalent to src.c as t then { t.p as p ... } though t is never made explicit in the case of subElements and cannot be referred to

  • transform(parameters): details how the content that is created is transformed from the source data using a specific transformation function. See below for the list of available functions. If no transform is provided, then the element is autocreated. It is an error if auto-created elements are primitive types, or have more than one possible type
  • as variable: a variable name may be assigned, which allows the created item to be re-used in the dependent rules
  • list_modes: control how elements that repeat are managed when the transform rule is evaluated. Values can be 'first', 'share', 'last' and 'single'. TODO: What do these do?

Transform statements may just contain an invocation of a transform function. In this case, a variable must be defined, and the created value is available in the variable for use in subsequent transformations.

Each time the rule is applied, the engine determines the value from the transforms, considers the list mode, if required and creates that specified content in the target instance. Within a target transform, the target statements are processed in order, so that a transform statement may refer to a variable defined by a prior transform statement.

The following list specifies that transforms that can be specified. Each transform takes one or more parameters:

Name Parameters Documentation FHIRPath equivalent
create type use the standard API to create a new instance of data. Where structure definitions have been provided, the type parameter must be a string which is a known type of a root element. Where they haven't, the application must know the name somehow n/a
copy source simply copy the source to the target as is (only allowed when the types in source and target match- typically for primitive types). In the concrete syntax, this is simply represented as the source variable, e.g. src.a = tgt.b n/a
truncate source, length source must be some stringy type that has some meaningful length property substring icon
escape source, format1, format2 Change the internal escaping of a string element. Note: this is not often needed, as mostly the escaping is done on the base format n/a
cast source, type? cast source from one type to another. target type can be left as implicit if there is one and only one target type known. . The default namespace for the type is 'FHIR' (see FHIRPath type specifiers icon) n/a
append source... source is element or string - just append them all together & (String concatenation) icon
translate source, map_uri, output use the translate operation. The source is some type of code or coded datatype, and the source and map_uri are passed to the translate operation. The output determines what value from the translate operation is used for the result of the operation (code, system, display, Coding, or CodeableConcept) %terminologies.translate()
reference source return a string that references the provided tree properly n/a
dateOp ?? Perform a date operation. Parameters to be documented n/a
uuid n/a Generate a random UUID (in lowercase). No Parameters n/a
pointer resource Return the appropriate string to put in a Reference that refers to the resource provided as a parameter related to resolve()
translate (varies)) translate(source, uri_of_map) - use the translate operation n/a
evaluate resource Execute the supplied FHIRPath expression and use the value returned by that. The 2nd parameter - FHIRPath expression - is evaluated in the context of the first parameter, and the result used as the value. If the outcome of the evaluation of the FHIRPath expression is an empty collection, no element is created in the target. If the outcome has a single value, the target is created with that value. If the outcome has more than one value, and the element is repeating, a separate target instance will be created for each value. If there is more than one value and the element is non-repeating this is treated as an error.
In the concrete syntax, there is a short hand for this operation, by supplying () around the parameter. In this case, there is no explicit context for the FHIRPath expression, and any context is implicit through references to existing variables such as $this.
n/a
cc (text) or (system. Code[, display]) Create a CodeableConcept from the parameters provided %factory.CodeableConcept()">
c system. Code[, display] Create a Coding from the parameters provided %factory.Coding()">
qty (text) or (value, unit, [system, code]) Create a quantity. Parameters = (text) or (value, unit, [system, code]) where text =s the natural representation e.g. [comparator]value[space]unit %factory.Quantity()">
id system, value[, type] Create an identifier. where type is a code from the identifier type value set %factory.Identifier()">
cp (value) or (system, value) Create a contact detail. If no system is provided, the system should be inferred from the content of the value %factory.ContactPoint()">

TODO: explain how optional parameters work with transforms (append only?), document list mode

The underlying type systems for source and target MAY define required implicit conversions, especially around how primitive values are handled and represented as strings (often required implicitly when executing the mapping rules).

As an example, see the FHIRPath specification implicit type conversions icon, which applies to source and/or target if they are resources (or content described by FHIR logical models).

In some cases, the transform statements will cause the automatic creation of objects in the target content. If the target model describes a choice of types for the automatically constructed object, then the engine raises an error. In such cases, an explicit create should be used:

  tgt.aa = create("x") as t_aa...

Once the source elements are evaluated, and any specified targets created, the engine has a set of variables that represent source and target contexts in which further mapping may occur. The set of variables includes those provided to the group that contains the rule, and those created by the application of the rule. For some created elements that are primitive types, that's the end of the road - there's nothing more to do with them. But if either or both the source and target types are complex, there are usually additional mapping rules that need to apply to the newly created variables.

Dependent rules specify what additional rules are evaluated when the rule is complete:

 .. then {
   .. other rules...
 }

When a rule contains other rules, the variables from the containing rules are all available to the contained rules. Alternatively, a rule can nominate other groups of rules from the same or an imported mapping. Each group is listed by name, and then a set of parameters are provided.

 .. then group(param, param)

The parameters provided must match the parameters required by the dependent group, in order. In addition, the mode of the variable must match - inputs that are targets must be target variables. Note, though, that target variables can be treated as source for a group.

Multiple groups may be specified, each separated by a comma. The last group invocation of the rule may be followed by a set of dependent rules. Dependent rules are evaluated after all nominated groups have been invoked.

Groups are resolved by name by looking through all the groups in all the available maps referenced by the uses (see above) statements. The name must be unique within the scope of these maps.

If no dependent rules are specified, and if the is only one source and target, and they both specify a variable, the rule can be written in an abbreviated form:

 src.element -> tgt.element;

This is implicitly the same as

  src.element as vvs -> tgt.element = create('type') as vvt then defaultMappingGroup(vvs, vvt)

Where the name of the type given as a parameter to 'create' and the group invoked by the 'then' are determined by the context of src.element and tgt.element and the selected default mapping group, as documented in the next section. Note that default mapping groups are only invoked when no dependent rules or explicit group invocations are specified.

This simple form is sometimes known as the identity transform, though strictly, the types of the source and the target may be different (e.g. different FHIR versions), but have the same properties and value domains. The simple transform (copy all properties as is recursively following the types) is sufficiently common that there's an even shorter form defined:

  src -> tgt: type, subtype, action, recorded;

This syntax is allowed as the first entry in the rules section, and is a purely syntactic short-cut for the longer form:

  src.type -> tgt.type;
  src.subtype -> tgt.subtype;
  src.action -> tgt.action;
  src.recorded -> tgt.recorded;

The primary attraction of this shorter form is that it allows the reader to focus on the parts of the transform that involve change, which is attractive in version to version changes where there is changes on only a few elements.

It is not necessary to explicitly invoke groups for each mapping. Instead groups can be declared to be the "default" mapping for a given source and target type. Groups acting as defaults have either <<types>> or <<type+>> in their declaration.

Groups marked with types are used by default when the engine encounters a mapping with a source and target type where the types match the source and target type of the group. Of course, there can be only one such group for each combination of source and target type for the engine to unambiguously determine which default group to invoke.

In addition to the above use, groups may be marked with type+. They will act like a default mapping group, just like types, but additionally they will be invoked when the target type is not fixed, i.e. when mapping to an element with a choice type. This means that a type+ group will be used as the default as long as the source type of the instance to map matches the source type of the group. Even so, the target will then always be taken to be the target type of the group.

The formal grammar for the mapping language, specified using ANTLR, can be found here.

Note that this grammar uses FHIRPath as an embedded syntax. Full details on FHIRPath and its grammar can be found here icon.

todo

This is the list of reserved keywords, which cannot be used as identifiers and names for variables, unless escaped.

map
uses
as
alias
imports
group
extends
default
where
check
log
then
true
false
types
type
first
not_first
last
not_last
only_one
share
single
source
target
queried
produced
conceptMap
prefix