Data Adapters¶

A common task for implementors is to replicate all data collected by NF to another system. While the Sparkplug/MQTT interface provides one way to do this, implementing others is relatively straightforward.

While several different mechanisms are available to synchronize data, the one we recommend uses the ObserveDataUpdates API.

The ObserveDataUpdates API is log-based: each data point collected by NF is assigned a unique, monotonic version. By remembering this version number, a client can easily return all data "after" the last value they received. Note that because the versions are assigned sequentially by NF, you are sure to get all newer data without needing to treat corner cases such as when an offline devices backloads historical data, as is the case when querying for time ranges from the timeseries store.

Making a request¶

When making a request, the earliest possible version is 0-0, so you can provide that to retrieve all available data. By default NF will keep data available through this API for around 30 days after they are added (regardless of the timestamp in the data). You can also set a limit value to set a batch size for how many records to return.

$ curl "localhost:8080/api/v1/point/updates/data?version=0-0&limit=100"

The result of this will be a list of objects which look like this:

{
 "uuid": "9152df58-3a44-3d30-951a-478f7fb33948",
 "version": "1626106800091-1",
 "value": {
  "ts": "2021-07-12T16:20:00.085855490Z",
  "unsigned": 4294967295
 }
}

The value field contains the actual data point with the "real" timestamp". For the next batch, take the last version received and request replace 0-0 in your request with that version.

While there is no hard limit to how large a batch you can request, you may experience timeouts if too many records are requested.

Obtaining metadata¶

In addition to the raw data values, NF also stores metadata about each point in the form of attribute dictionaries. There are a number of ways to obtain metadata, including ObservePointsUpdate, which provides a similar log-based API for metadata changes; GetPoints which provides indexed text search over all points; and GetPointsById which looks up metadata by UUID.

The simplest way to get metadata when replicating data is to simply set withMetadata=true when using this API. Then, the current set of attributes will be provided with each new value.

$ curl "localhost:8080/api/v1/point/updates/data?version=0-0&limit=100&withMetadata=true"