Data Collection reads Devices from MTConnect Agents, stores their data in a database, and publishes it to an MQTT broker. It's run by the Collection Worker, and licensed per Device (see Collection Licenses).
How Devices Are Read
- The Device model is read with a Probe request.
- Observations are streamed from the Agent with Current and Sample requests, so new data is received as soon as the Agent has it. The Device's Stream Interval and Stream Heartbeat set how often the Agent sends it.
- The position in each Agent's buffer is saved, so after a restart Collection resumes where it left off instead of missing data or reading it twice. Reset Position on a Device's Collection tab reads it again from the Agent.
- MTConnect Assets are read after each Probe, and whenever an
ASSET_CHANGEDorASSET_REMOVEDObservation is received. - Each Device's Connection Status, and any errors from the Agent, are recorded (see Monitoring Collection).
SHDR Devices, and Devices with a saved Model, are read through their Agent at the Agent Hub, so they're collected the same way, with a single connection to the source.
When a Device Is Collected
A Device is collected once:
- Collection is turned on for it (Collect data when adding it, Add to Collection on its Collection tab, or Turn On Collection for its Group).
- It has a License (a free trial works too). The Collection Worker checks Licenses every minute, and stops a Device whose License expires or is unassigned.
Its data is then sent to each of its Profiles. A Device without a Profile shows No Profile: it's still read and its Connection Status is shown, but its data isn't stored or published anywhere.
Where the Data Goes
A Device has the Profiles selected on its Collection tab (or on the Profile's Devices tab), plus those of its Group. Each Profile has its own Filter, so a Device can, for example, store every DataItem in Tables while only publishing Conditions as Messages.
- A destination used by several of a Device's Profiles only receives the data once.
- A destination a running Device gains is sent its model and current values within a few seconds, without restarting the Device. Assets and earlier Observations aren't sent until Reset Position is used.
- Observations are written to databases in batches, through a Buffer that's kept on disk when the database can't keep up or can't be reached (see Buffers).
What's Stored
| Data | Tables | Messages |
|---|---|---|
| Device models (Devices, Components, DataItems, and their Configurations) | Yes | Yes |
| Current values of each DataItem | Yes | Yes (retained) |
| Historical Observations (Samples, Events, Conditions, Tables, DataSets, and TimeSeries) | Yes | As they're collected |
| MTConnect Assets (ex. Cutting Tools, Files, Raw Materials) | Yes (Store Assets) | |
| Connection Status | Yes, with history | Yes |
| Observation Statistics | Yes | Yes |
See Database Tables and MQTT Messages for where each is kept.