TestScript is a resource that specifies a suite of tests against a FHIR server implementation to determine compliance against the FHIR specification.

An absolute URL that is used to identify this Test Script. This SHALL be a URL, SHOULD be globally unique, and SHOULD be an address at which this Test Script is (or will be) published.

The identifier that is used to identify this version of the TestScript. This is an arbitrary value managed by the TestScript author manually.

There may be multiple resource versions of the TestScript that have this same identifier. The resource version id will change for technical reasons, whereas the stated version number needs to be under the author's control.

A free text natural language name identifying the TestScript.

Not expected to be globally unique.

The status of the TestScript.

Allows filtering of TestScripts that are appropriate for use vs. not.

Identifier for the TestScript assigned for external purposes outside the context of FHIR.

This TestScript was authored for testing purposes (or education/evaluation/marketing), and is not intended to be used for genuine usage.

Allows filtering of TestScripts that are appropriate for use vs. not.

The name of the individual or organization that published the Test Script.

Helps establish the "authority/credibility" of the Test Script. May also allow for contact.

Usually an organization, but may be an individual. This item SHOULD be populated unless the information is available from context.

Contacts to assist a user in finding and communicating with the publisher.

May be a web site, an email address, a telephone number (tel:), etc.

The name of an individual to contact regarding the Test Script.

If there is no named individual, the telecom is for the organization as a whole.

Contact details for individual (if a name was provided) or the publisher.

The date this version of the test tcript was published. The date must change when the business version changes, if it does, and it must change if the status code changes. In addition, it should change when the substantive content of the test cases change.

Additional specific dates may be added as extensions.

A free text natural language description of the TestScript and its use.

This field can be used for things such as why the TestScript was written, comments about misuse, instructions for clinical use and interpretation, literature references, examples from the paper world, etc. It is not a rendering of the TestScript as conveyed in TestScript.text. This item SHOULD be populated unless the information is available from context.

The content was developed with a focus and intent of supporting the contexts that are listed. These terms may be used to assist with indexing and searching of Test Scripts.

Assist in searching for appropriate content.

Explains why this Test Script is needed and why it's been constrained as it has.

This element does not describe the usage of the Test Script (that's done in comments), rather it's for traceability of why the element is either needed or why the constraints exist as they do. This may be used to point to source materials or specifications that drove the structure of this data element.

A copyright statement relating to the Test Script and/or its contents. Copyright statements are generally legal restrictions on the use and publishing of the details of the constraints and mappings.

An abstract server used in operations within this test script in the origin element.

The purpose of this element is to define the profile of an origin element used elsewhere in the script. Test engines could then use the origin-profile mapping to offer a filtered list of test systems that can serve as the sender for the interaction.

Abstract name given to an origin server in this test script. The name is provided as a number starting at 1.

A given origin index (e.g. 1) can appear only once in the list (e.g. Origin 1 cannot be specified twice ... once as FormFiller and and again as FormProcessor within the same script as that could get confusing during test configuration).

Different origin indices could play the same actor in the same test script (e.g. You could have two different test systems acting as Form-Filler).

The origin indices provided elsewhere in the test script must be one of these origin indices.

The type of origin profile the test system supports.

Must be a "sender"/"client" profile.

An abstract server used in operations within this test script in the destination element.

The purpose of this element is to define the profile of an destination element used elsewhere in the script. Test engines could then use the destination-profile mapping to offer a filtered list of test systems that can serve as the receiver for the interaction.

Abstract name given to a destination server in this test script. The name is provided as a number starting at 1.

A given destination index (e.g. 1) can appear only once in the list (e.g. Destination 1 cannot be specified twice ... once as Form-Manager and and again as Form-Processor within the same script as that could get confusing during test configuration).

Different destination indices could play the same actor in the same test script (e.g. You could have two different test systems acting as Form-Manager).

The destination indices provided elsewhere in the test script must be one of these destination indices.

The type of destination profile the test system supports.

Must be a "receiver"/"server" profile.

The required capability must exist and are assumed to function correctly on the FHIR server being tested.

A link to the FHIR specification that this test is covering.

URL to a particular requirement or feature within the FHIR specification.

Short description of the link.

Capabilities that must exist and are assumed to function correctly on the FHIR server being tested.

When the metadata capabilities section is defined at TestScript.metadata or at TestScript.setup.metadata, and the server's conformance statement does not contain the elements defined in the minimal conformance statement, then all the tests in the TestScript are skipped. When the metadata capabilities section is defined at TestScript.test.metadata and the server's conformance statement does not contain the elements defined in the minimal conformance statement, then only that test is skipped. The "metadata.capabilities.required" and "metadata.capabilities.validated" elements only indicate whether the capabilities are the primary focus of the test script or not. The do not impact the skipping logic. Capabilities whose "metadata.capabilities.validated" flag is true are the primary focus of the test script.

Whether or not the test execution will require the given capabilities of the server in order for this test script to execute.

Whether or not the test execution will validate the given capabilities of the server in order for this test script to execute.

Description of the capabilities that this test script is requiring the server to support.

Which origin server these requirements apply to.

Which server these requirements apply to.

Links to the FHIR specification that describes this interaction and the resources involved in more detail.

Minimum conformance required of server for test script to execute successfully. If server does not meet at a minimum the reference conformance definition, then all tests in this script are skipped.

The conformance statement of the server has to contain at a minimum the contents of the reference pointed to by this element.

Fixture in the test script - by reference (uri). All fixtures are required for the test script to execute.

Whether or not to implicitly create the fixture during setup. If true, the fixture is automatically created on each server being tested during setup, therefore no create operation is required for this fixture in the TestScript.setup section.

Whether or not to implicitly delete the fixture during teardown If true, the fixture is automatically deleted on each server being tested during teardown, therefore no delete operation is required for this fixture in the TestScript.teardown section.

Reference to the resource (containing the contents of the resource needed for operations).

See http://hl7-fhir.github.io/resourcelist.html for complete list of resource types.

Reference to the profile to be used for validation.

See http://hl7-fhir.github.io/resourcelist.html for complete list of resource types.

Variable is set based either on element value in response body or on header field value in the response headers.

Variables would be set based either on XPath/JsonPath expressions against fixtures (static and response), or headerField evaluations against response headers. If variable evaluates to nodelist or anything other than a primitive value, then test engine would report error. Variables would be used to perform clean replacements in "operation.params", "operation.requestHeader.value", and "operation.url" element values during operation calls and in "assert.value" during assertion evaluations. This limits the places that test engines would need to look for placeholders "${}". Variables are scoped to the whole script. They are NOT evaluated at declaration. They are evaluated by test engine when used for substitutions in "operation.params", "operation.requestHeader.value", and "operation.url" element values during operation calls and in "assert.value" during assertion evaluations. See example testscript-search.xml.

Descriptive name for this variable.

Placeholders would contain the variable name wrapped in ${} in "operation.params", "operation.requestHeader.value", and "operation.url" elements. These placeholders would need to be replaced by the variable value before the operation is executed.

A default, hard-coded, or user-defined value for this variable.

The purpose of this element is to allow for a pre-defined value that can be used as a default or as an override value. Test engines can optionally use this as a placeholder for user-defined execution time values.

Will be used to grab the HTTP header field value from the headers that sourceId is pointing to.

If headerField is defined, then the variable will be evaluated against the headers that sourceId is pointing to. If path is defined, then the variable will be evaluated against the fixture body that sourceId is pointing to. It is an error to define both headerField and path.

XPath or JSONPath against the fixture body. When variables are defined, either headerField must be specified or path, but not both.

If headerField is defined, then the variable will be evaluated against the headers that sourceId is pointing to. If path is defined, then the variable will be evaluated against the fixture body that sourceId is pointing to. It is an error to define both headerField and path.

Fixture to evaluate the XPath/JSONPath expression or the headerField against within this variable.

This can be a statically defined fixture (at the top of the testscript) or a dynamically set fixture created by responseId of the action.operation element.

Assert rule to be used in one or more asserts within the test script.

Each rule should be treated by test engines as one assertion regardless of how many assertions are contained within the external rule template. The impact of negative rule evaluation on test script execution is the same as an assertion failure which is descibed elsewhere in the TestScript resource.

Reference to the resource (containing the contents of the rule needed for assertions).

Each rule template can take one or more parameters for rule evaluation.

The parameter value can be dynamic at runtime.

Descriptive name for this parameter that matches the external assert rule parameter name.

The external rule template would be looking for the parameter by this name.

The explict or dynamic value for the parameter that will be passed on to the external rule template.

This value can be overwritten by the assert.rule.param.value i.e. TestScript.rule.param.value will be used if assert.rule.param.value is not specified. The param value can be a string-representation of a number, string, or boolean that is expected. Test engines do have to look for placeholders (${}) and replace the variable placeholders with the variable values at runtime before supplying this value to the external rule template.

Contains one or more rules. Offers a way to group rules so assertions could reference the group of rules and have them all applied.

Each rule within a ruleset should be treated by test engines as one assertion regardless of how many assertions are contained within the external rule template. The impact of negative rule evaluation on test script execution is the same as an assertion failure which is descibed elsewhere in the TestScript resource.

Reference to the resource (containing the contents of the ruleset needed for assertions).

Id of the referenced rule within the external ruleset template.

This qualifies each param name so that a parameter with the same name can be used differently by the different rules with the ruleset.

Each rule template can take one or more parameters for rule evaluation.

The parameter value can be dynamic at runtime.

Descriptive name for this parameter that matches the external assert ruleset rule parameter name.

The external rule template would be looking for the parameter by this name.

The value for the parameter that will be passed on to the external ruleset rule template.

This value can be overwritten by the assert.ruleset.rule.param.value i.e. TestScript.ruleset.rule.param.value will be used if assert.ruleset.rule.param.value is not specified. The param value can be a string-representation of a number, string, or boolean that is expected. Test engines do have to look for placeholders (${}) and replace the variable placeholders with the variable values at runtime before supplying this value to the external rule template.

A series of required setup operations before tests are executed.

Capabilities that must exist and are assumed to function correctly on the FHIR server being tested.

Action would contain either an operation or an assertion.

An action should contain either an operation or an assertion but not both. It can contain any number of variables.

The operation to perform.

Server interaction or operation type.

See http://hl7-fhir.github.io/http.html for list of server interactions.

The type of the resource. See http://hl7-fhir.github.io/resourcelist.html.

If "url" element is specified, then "targetId", "params", and "resource" elements will be ignored as "url" element will have everything needed for constructing the request url. If "params" element is specified, then "targetId" element is ignored. For FHIR operations that require a resource (e.g. "read" and "vread" operations), the "resource" element must be specified when "params" element is specified. If "url" and "params" elements are absent, then the request url will be constructed from "targetId" fixture if present. For "read" operation, the resource and id values will be extracted from "targetId" fixture and used to construct the url. For "vread" and "history" operations, the versionId value will also be used.

The label would be used for tracking/logging purposes by test engines.

This has no impact on the verification itself.

The description would be used by test engines for tracking and reporting purposes.

This has no impact on the verification itself.

The content-type or mime-type to use for RESTful operation in the 'Accept' header.

If this is specified, then test engine shall set the 'Accept' header to the corresponding value. If 'xml' is specified, then 'Accept' header of 'application/xml+fhir' will be set. If 'json' is specified, then 'application/json+fhir' will be used. If you'd like to explicitly set the 'Accept' to some other value then use the 'requestHeader' element.

The content-type or mime-type to use for RESTful operation in the 'Content-Type' header.

If this is specified, then test engine shall set the 'Content-Type' header to the corresponding value. If 'xml' is specified, then 'Content-Type' header of 'application/xml+fhir' will be set. If 'json' is specified, then 'application/json+fhir' will be used. If you'd like to explicitly set the 'Content-Type' to some other value then use the 'requestHeader' element.

The server where the request message is destined for. Must be one of the server numbers listed in TestScript.destination section.

If multiple TestScript.destination elements are defined and operation.destination is undefined, test engine will error as it cannot determine what destination to use for the exchange.

Whether or not to implicitly send the request url in encoded format. The default is true to match the standard RESTful client behavior. Set to false when communicating with a server that does not support encoded url paths.

The server where the request message originates from. Must be one of the server numbers listed in TestScript.origin section.

If absent, test engine will send the message. When present, test engine will not send the request message but will wait for the request message to be sent from this origin server.

Path plus parameters after [type]. Used to set parts of the request URL explicitly.

If "url" element is specified, then "targetId", "params", and "resource" elements will be ignored as "url" element will have everything needed for constructing the request url. If "params" element is specified, then "targetId" element is ignored. For FHIR operations that require a resource (e.g. "read" and "vread" operations), the "resource" element must be specified when "params" element is specified. If "url" and "params" elements are absent, then the request url will be constructed from "targetId" fixture if present. For "read" operation, the resource and id values will be extracted from "targetId" fixture and used to construct the url. For "vread" and "history" operations, the versionId value will also be used. Test engines would append whatever is specified for "params" to the URL after the resource type without tampering with the string (beyond encoding the URL for HTTP). The "params" element does not correspond exactly to "search parameters". Nor is it the "path". It corresponds to the part of the URL that comes after the [type] (when "resource" element is specified); e.g. It corresponds to "/[id]/history/[vid] {?format=[mime-type]}" in the following operation: GET [base]/[type]/[id]/history/[vid] {?format=[mime-type]} Test engines do have to look for placeholders (${}) and replace the variable placeholders with the variable values at runtime before sending the request.

Header elements would be used to set HTTP headers.

This gives control to test-script writers to set headers explicitly based on test requirements. It will allow for testing using: - "If-Modified-Since" and "If-None-Match" headers. See http://hl7-fhir.github.io/http.html#2.1.0.5.1 - "If-Match" header. See http://hl7-fhir.github.io/http.html#2.1.0.11 - Conditional Create using "If-None-Exist". See http://hl7-fhir.github.io/http.html#2.1.0.13.1 - Invalid "Content-Type" header for negative testing. - etc.

The HTTP header field e.g. "Accept".

If header element is specified, then field is required.

The value of the header e.g. "application/xml".

If header element is specified, then value is required. No conversions will be done by Test Engine e.g. "xml" to "application/xml+fhir". The values will be set in HTTP headers "as-is". Test engines do have to look for placeholders (${}) and replace the variable placeholders with the variable values at runtime before sending the request.

The fixture id (maybe new) to map to the response.

If a responseId is supplied, and the server responds, then the resulting response (both headers and body) is mapped to the fixture ID (which may be entirely new and previously undeclared) designated by "responseId". If responseId is not specified, it is the Test Engine's responsibility to store the response and use it as sourceId in subsequent assertions when assertion path and/or headerField is specified and sourceId is not specified.

The id of the fixture used as the body of a PUT or POST request.

Id of fixture used for extracting the [id], [type], and [vid] for GET requests.

If "url" element is specified, then "targetId", "params", and "resource" elements will be ignored as "url" element will have everything needed for constructing the request url. If "params" element is specified, then "targetId" element is ignored. For FHIR operations that require a resource (e.g. "read" and "vread" operations), the "resource" element must be specified when "params" element is specified. If "url" and "params" elements are absent, then the request url will be constructed from "targetId" fixture if present. For "read" operation, the resource and id values will be extracted from "targetId" fixture and used to construct the url. For "vread" and "history" operations, the versionId value will also be used.

Complete request URL.

Used to set the request URL explicitly. If "url" element is defined, then "targetId", "resource", and "params" elements will be ignored. Test engines would use whatever is specified in "url" without tampering with the string (beyond encoding the URL for HTTP). Test engines do have to look for placeholders (${}) and replace the variable placeholders with the variable values at runtime before sending the request.

Evaluates the results of previous operations to determine if the server under test behaves appropriately.

In order to evaluate an assertion, the request, response, and results of the most recently executed operation must always be maintained by the test engine.

The label would be used for tracking/logging purposes by test engines.

This has no impact on the verification itself.

The description would be used by test engines for tracking and reporting purposes.

This has no impact on the verification itself.

The direction to use for the assertion.

Id of fixture used to compare the "sourceId/path" evaluations to.

The id of the fixture used to make comparisons to.

XPath or JSONPath expression against fixture used to compare the "sourceId/path" evaluations to.

The XPath or JSONPath expression to be evaluated against the expected fixture to compare to. Ignored if "assert.value" is used. The evaluation will be done before the assertion is evaluated.

The content-type or mime-type to use for RESTful operation in the 'Content-Type' header.

If this is specified, then test engine shall confirm that the content-type of the last operation's headers is set to this value. If "assert.sourceId" element is specified, then the evaluation will be done against the headers mapped to that sourceId (and not the last operation's headers). If 'xml' is specified, then 'Content-Type' header of 'application/xml+fhir' will be confirmed. If 'json' is specified, then 'application/json+fhir' will be used. If you'd like to have more control over the string, then use 'assert.headerField' instead.

The HTTP header field name e.g. 'Location'.

If "headerField" is specified then "value" must be specified. If "sourceId" is not specified, then "headerField" will be evaluated against the last operation's response headers. Test engines are to keep track of the last operation's response body and response headers.

The ID of a fixture. Asserts that the response contains at a minimum the fixture specified by minimumId.

Asserts that the response contains all the element/content in another fixture pointed to by minimumId. This can be a statically defined fixture or one that is dynamically set via responseId.

Whether or not the test execution performs validation on the bundle navigation links.

Asserts that the Bundle contains first, last, and next links.

The operator type.

Operators are useful especially for negative testing. If operator is not specified, then the "equals" operator is assumed; e.g. <code> <assert> <operator value="in" /> <responseCode value="200,201,204" /> </assert> <assert> <operator value="notEquals" /> <response value="okay"/> </assert> <assert> <operator value="greaterThan" /> <responseHeader> <field value="Content-Length" /> <value value="0" /> <responseHeader/> </assert> </code>.

The XPath or JSONPath expression to be evaluated against the fixture representing the response received from server.

If both "path" and "fixtureId" are specified, then the path will be evaluated against the responseBody mapped to the fixtureId. If "path" is specified and "fixtureId" is not, then the path will be evaluated against the responseBody of the last operation. Test engines are to store the response body and headers of the last operation at all times for subsequent assertions.

The type of the resource. See http://hl7-fhir.github.io/resourcelist.html.

This will be expected resource type in response body e.g. in read, vread, search, etc. See http://hl7-fhir.github.io/resourcelist.html for complete list of resource types; e.g. <assert > <resourceType value="Patient" </assert>.

okay | created | noContent | notModified | bad | forbidden | notFound | methodNotAllowed | conflict | gone | preconditionFailed | unprocessable.

This is a shorter way of achieving similar verifications via "assert.responseCode". If you need more control, then use "assert.responseCode" e.g. <assert> <contentType value="json" /> <response value="okay"/> </assert>.

The value of the HTTP response code to be tested.

To be used with "operator" attribute value. Asserts that the response code equals this value if "operator" is not specified. If the operator is "in" or "notIn" then the responseCode would be a comma-separated list of values e.g. "200,201". Otherwise, it's expected to be a numeric value. If "fixture" is not specified, then the "responseBodyId" value of the last operation is assumed.

The TestScript.rule id value this assert will evaluate.

Each rule should get evaluated by test engines as one assertion regardless of how many assertions are contained within the external rule template. The impact of negative rule evaluation on test script execution is the same as an assertion failure which is descibed elsewhere in the TestScript resource.

Each rule template can take one or more parameters for rule evaluation.

The parameter value can be dynamic at runtime.

Descriptive name for this parameter that matches the external assert rule parameter name.

The external rule template would be looking for the parameter by this name.

The value for the parameter that will be passed on to the external rule template.

This value overwrites the value (if any) specified in TestScript.rule.param.value. The param value can be a string-representation of a number, string, or boolean that is expected. Test engines do have to look for placeholders (${}) and replace the variable placeholders with the variable values at runtime before supplying this value to the external rule template.

The TestScript.ruleset id value this assert will evaluate.

Each rule within a ruleset should get evaluated by test engines as a separate assertion. The impact of negative rule evaluation on test script execution is the same as an assertion failure which is descibed elsewhere in the TestScript resource. If the first rule within the ruleset results in a failed assertion, then test engines do not have to evaluate the rest of the rules within the ruleset.

Id of the referenced rule within the external ruleset template.

This qualifies each param name so that a parameter with the same name can be used differently by the different rules with the ruleset.

Each rule template can take one or more parameters for rule evaluation.

The parameter value can be dynamic at runtime.

Descriptive name for this parameter that matches the external assert ruleset rule parameter name.

The external rule template would be looking for the parameter by this name.

The value for the parameter that will be passed on to the external ruleset rule template.

This value overwrites the value (if any) specified in TestScript.ruleset.rule.param.value. The param value can be a string-representation of a number, string, or boolean that is expected. Test engines do have to look for placeholders (${}) and replace the variable placeholders with the variable values at runtime before supplying this value to the external rule template.

Fixture to evaluate the XPath/JSONPath expression or the headerField against.

This can be a statically defined fixture (at the top of the testscript) or a dynamically set fixture created by responseId of the action.operation element.

The ID of the Profile to validate against.

The ID of a Profile fixture. Asserts that the response is valid according to the Profile specified by validateProfileId.

The value to compare to.

The string-representation of a number, string, or boolean that is expected. Test engines do have to look for placeholders (${}) and replace the variable placeholders with the variable values at runtime before comparing this value to the actual value.

Whether or not the test execution will produce a warning only on error for this assert.

If this element is specified and it is true, then assertion failures can be logged by test engine but should not stop the test script execution from proceeding. There are likely cases where the spec is not clear on what should happen. If the spec says something is optional (maybe a response header for example), but a server doesn’t do it, we could choose to issue a warning.

A test in this script.

The name of this test used for tracking/logging purposes by test engines.

A short description of the test used by test engines for tracking and reporting purposes.

Capabilities that must exist and are assumed to function correctly on the FHIR server being tested.

Action would contain either an operation or an assertion.

An action should contain either an operation or an assertion but not both. It can contain any number of variables.

An operation would involve a REST request to a server.

Evaluates the results of previous operations to determine if the server under test behaves appropriately.

In order to evaluate an assertion, the request, response, and results of the most recently executed operation must always be maintained by the test engine.

A series of operations required to clean up after the all the tests are executed (successfully or otherwise).

The teardown action will only contain an operation.

An action should contain either an operation or an assertion but not both. It can contain any number of variables.

An operation would involve a REST request to a server.