Polling is a mechanism for conveying new data to a data consumer as (or shortly after) the data is created or updated without requiring the data source to be aware of the specific needs of the data consumer. This is accomplished by the data consumer repeatedly querying the data source to see if there is new data. The queries occur at period intervals of an agreed frequency. The frequency needs to be often enough that the time between when the relevant data is created and when the data consumer receives it is sufficiently short for the data consumer's needs. However, it needs to be infrequent enough that the data source's resources are not over-taxed by the repeated queries. In addition to being narrowly tuned to only include the specific resources that are of interest to the data consumer, the queries will generally also filter to only retrieve data if it has been created or changed since the previous query was run. Provided the relevant tables are indexed by _lastUpdated, this should also dramatically increase the efficiency of performing the query and minimize the bandwidth consumed.

Polling is generally only used when Subscription is not available or feasible. Guidance on whether to use polling or subscription can be found here

There are two principle ways to poll for data - polling a single record or polling across records.

Polling a single record

This approach is used when the data consumer is only interested in a single resource that already exists. For example, when monitoring a Task for progress or to see if it is complete (e.g. for Task-based exchanges). Rather than performing a search, the data consumer simply executes a read. To be more efficient, it uses the If-modified-since header specifying the _lastUpdated value of the most recent version it has retrieved. This ensures that no data will be returned at all if the record has not changed since it was last looked at.

Note that while the 'read' mechanism is extremely efficient, separate requests must be made for each record being monitored. If the data consumer is tracking numerous records on the data source, it might be more efficient for both systems to use the query polling mechanism instead.

Polling across records

This approach uses a search mechanism to find the relevant records. It can be used when monitoring for the creation of new records, when the ids of the records that are relevant may not be known in advance (because changes to values such as status might cause records to fall into or out of the relevant pool), or when the number of relevant records is simply too large for the single-record polling approach to be feasible. Typically the "across-record" approach will use the standard search query mechanism. However, any search mechanism that returns existing could potentially be used. Guidance on selecting the appropriate one can be found here. It can be possible to execute multiple polling queries simultaneously using the batch approach. Whatever query mechanism is used, when polling only synchronous queries are appropriate.

If-modified-since is not available when performing queries. Instead, whatever query mechanism is used, it should include the filter of &_lastUpdated=gt[sometimestamp] where [sometimestamp] is the most recent meta.lastUpdated value retrieved in previous poll requests.