# Operate

> Logs, metrics, health checks, reloading configuration, and the durable buffer of a running Agent or Adapter.

URL: https://www.trakhound.com/docs/mtconnect-net/configure/operate

This page covers the operational surface of a running agent or adapter—logs (where they land, what each logger reports), metrics (observation rate, asset count), health checks, reload + restart semantics, and durable-buffer handling. It is the page an on-call operator opens at 03:00 when something is unhappy.

[Connect a Consumer](https://www.trakhound.com/docs/mtconnect-net/configure/consumer) covers the data plane. This page covers the control plane—everything the operator (not the consumer) needs to know.

## Logs

Logging is provided by NLog. The shipped `NLog.config` next to the agent / adapter executable defines five named loggers, each writing to its own daily-rolled file under `logs/`:

| Logger | Source | File pattern |
| --- | --- | --- |
| `application-logger` | Top-level lifecycle (startup, shutdown, configuration load). | `logs/application-<YYYY-MM-DD>.log` |
| `agent-logger` | Agent broker internals (device add, observation enqueue, buffer events). | `logs/agent-<YYYY-MM-DD>.log` |
| `agent-validation` | Input validation issues (rejected observations, type mismatches). | `logs/agent-validation-<YYYY-MM-DD>.log` |
| `agent-metrics` | Periodic agent-metrics emitter (observation rate, asset rate). | Routed to its own file when configured; otherwise rolls into `agent-`. |
| `modules.<module-id>` | One file per loaded module (the logger name is `modules.` prefixed onto the module key—`modules.mqtt-relay`, `modules.shdr-adapter`, etc.). | `logs/modules.<module-id>-<YYYY-MM-DD>.log` |
| `processors.<processor-id>` | One file per loaded processor (e.g. `processors.agent-processor-python`). | `logs/processors.<processor-id>-<YYYY-MM-DD>.log` |

Every file rolls daily and keeps 90 archives by default. Edit `NLog.config` to raise the archive count, change the layout, or add a Syslog / TCP / cloud target—the shipped configuration is intended as a working starting point, not a frozen contract.

### Raising the Console Log Level

The `debug` and `trace` CLI verbs (see [Run](https://www.trakhound.com/docs/mtconnect-net/configure/run)) raise the console-target log level on a single foreground run without editing `NLog.config`. For a service deployment, edit `NLog.config` directly—to pick up edits without a process restart, set `autoReload="true"` on the `<nlog>` root element (this is NLog's hot-reload lever; the shipped configuration leaves it off so an edit-and-restart cycle is the default).

### NLog at a Glance

The shipped file targets use the layout:

```
${longdate}|${event-properties:item=EventId_Id:whenEmpty=0}|${uppercase:${level}}|${logger}|${message} ${exception:format=tostring}
```

Pipe-separated; columns are time, event-ID, level, logger, message + optional exception. The format ingests cleanly into typical log-aggregation pipelines (Elastic / Loki / Datadog)—point a parser at the `logs/` directory and parse on the pipe character.

## Metrics

The agent runs a built-in metrics timer in the `MTConnect.Agents.Metrics` namespace. On each tick it reads the current observation count, the current asset count, the rolling-window averages, and emits debug-level lines through the `agent-metrics` logger:

```
agent-metrics | Debug | Observations - Delta for last 10 seconds: 423
agent-metrics | Debug | Observations - Average for last 5 minutes: 41.78
agent-metrics | Debug | Assets - Delta for last 10 seconds: 0
agent-metrics | Debug | Assets - Average for last 5 minutes: 0.13
```

The metrics emitter is gated by a single switch—`enableMetrics: true` in `agent.config.yaml` (the shipped default; set to `false` to suppress the periodic emits entirely). The tick interval and the rolling-window length are fixed at construction time by the agent host; the YAML surface does not expose them today.

For machine-readable metrics, query the agent's `/probe` self-describing Agent Device when `enableAgentDevice: true` is set. The Agent Device emits `AVAILABILITY`, `ASSET_CHANGED`, `ASSET_REMOVED`, `MTCONNECT_VERSION`, and a battery of internal observations. Probes / current / sample work on the Agent Device the same way as any other device:

```sh
curl -s 'http://agent.local:5000/Agent/current'
```

## Health Checks

For load-balancer health probes, the simplest signal is an HTTP `GET /probe` on the agent's port. A `200 OK` with a `<MTConnectDevices>` body means the HTTP server is up and the agent loaded at least one device. A `503 Service Unavailable` with an `<MTConnectError>` envelope means the agent is running but in an error state.

A liveness-only probe (server is up, but the buffer / device-model state is unverified) can hit the agent's static-root handler:

```sh
curl -s -o /dev/null -w '%{http_code}' http://agent.local:5000/
```

A `200` confirms the HTTP server is responding without exercising any device-model lookups.

## Reload and Restart

The agent supports two restart shapes:

1. **Soft reload**—set `monitorConfigurationFiles: true` in `agent.config.yaml`. The agent watches `agent.config.yaml` and every `Devices.xml` file under `devices:` and reloads automatically when any of them changes (debounced by `configurationFileRestartInterval`, default 2 seconds). The HTTP server keeps its port; existing long-poll connections are gracefully closed and reconnect from the consumer side.
2. **Hard restart**—the OS-level mechanism: `systemctl restart mtconnect-agent` on Linux; `sc stop / sc start MTConnect.NET-Agent` (or the bundled `MTConnect.NET-Agent stop` + `start` verbs) on Windows.

The soft reload preserves the in-memory buffer; the hard restart loses it unless `durable: true` is set (see below).

When `monitorConfigurationFiles: true` is enabled, an edit to `Devices.xml` triggers a `DeviceConfigurationFileWatcher` event; the agent re-parses the file, validates the result against the loaded XSD, and atomically swaps the device tree. A parse failure leaves the previous device tree in place and surfaces the error through `application-logger`.

## Durable Buffer

When `durable: true` is set, the agent writes its observation buffer pages to `durableBufferPath` (default `buffer/` next to the executable). On startup the agent restores the buffer, preserves the `instanceId`, and resumes from the last persisted `nextSequence`. A consumer paginating across a restart sees a continuous sequence-cursor rather than a reset.

Operational notes:

- The buffer directory grows with `observationBufferSize` \* the per-observation page size; provision disk accordingly. The shipped default of `observationBufferSize: 131072` typically occupies tens of megabytes.
- A corrupted buffer (truncated page, disk-full mid-write) prevents startup. The `reset` CLI verb (see [Run](https://www.trakhound.com/docs/mtconnect-net/configure/run)) wipes the buffer and clears the instance state so the agent can boot fresh.
- The buffer is not backed up to off-machine storage by default. For deployments where buffer survival across machine failure matters, snapshot the `buffer/` directory through the usual filesystem-snapshot machinery (LVM, ZFS, EBS snapshots).

## Common Operational Patterns

- **Tail the agent log in production**: `tail -F logs/agent-$(date +%F).log` (Linux) or `Get-Content -Tail 50 -Wait logs\agent-(Get-Date -Format yyyy-MM-dd).log` (PowerShell).
- **Tail validation rejections only**: `tail -F logs/agent-validation-$(date +%F).log`. A spike here signals a misconfigured adapter or a Devices.xml drift.
- **Tail a specific module or processor**: `tail -F logs/modules.<module-id>-$(date +%F).log` (e.g. `logs/modules.http-server-$(date +%F).log`); same shape for processors via `logs/processors.<processor-id>-$(date +%F).log`.
- **Diagnose a single module without restarting**: enable the module's debug level by editing `NLog.config`'s per-module rule. Hot-reload requires `autoReload="true"` on the `<nlog>` root; otherwise restart the agent for the new rule to apply.
- **Test the agent's reachability from inside a Docker network**: `docker exec mtc-agent curl -sf http://localhost:5000/probe > /dev/null`. A non-zero exit means the in-container HTTP server is down even if the host-side port mapping appears healthy.

## See Also

- [Install](https://www.trakhound.com/docs/mtconnect-net/configure/install)—installer / package bring-up that this page assumes is complete.
- [Configure an Adapter](https://www.trakhound.com/docs/mtconnect-net/configure/adapter-config)—adapter-side `adapter.config.yaml` reference; an unhappy adapter is the most common driver of an unhappy agent.
- [Run](https://www.trakhound.com/docs/mtconnect-net/configure/run)—starting and stopping the agent.
- [Connect a Consumer](https://www.trakhound.com/docs/mtconnect-net/configure/consumer)—the consumer side of the running agent.
- [Configure an Agent](https://www.trakhound.com/docs/mtconnect-net/configure/agent-config)—every `agent.config.yaml` key including the operational ones (`durable`, `metrics`, `monitorConfigurationFiles`).
