Monitoring and metrics

Per-channel metrics

Every channel exposes the following in the interface and over the HTTP API.

Wire bitrate

The rate of transport-stream data actually leaving the interface for that channel’s destination, measured by packet capture rather than reported by the engine. This is the one figure to trust. It is engine-agnostic, and it is the only measurement taken outside the process producing the stream, so it cannot be fooled by an engine that believes it is working.

Frames per second

Decoded frame rate. Remux engine only.

Playback speed

Progress through the source relative to real time. A sustained value below 1.0 means the channel cannot keep up. Remux engine only.

Encoder bitrate

The rate the engine believes it is producing, as distinct from the rate observed on the wire. Remux engine only.

Uptime

Time since the current engine process started. It resets on every restart, which makes a short uptime on a channel you have not touched a signal in itself.

Viewers

Number of televisions currently joined to the destination group, and their addresses. See On-demand streaming for how this is measured and when it is marked degraded.

Round-trip time to source

Latency to the origin host. Useful for separating “the origin is slow” from “the path to the origin is slow”.

Channel log

A per-channel buffer of the last 1000 entries or 24 hours, whichever is reached first, readable in the interface without shell access to the appliance.

Do not build alerts on frames-per-second or playback speed

These come from the remux engine’s own reporting. The pass-through engine never decodes video — that is the entire point of it — so it produces no frame rate and no playback speed. Those fields read n/a for such channels, and that is the correct value, not a fault.

Any watchdog keyed on them will read n/a as zero and restart a perfectly healthy channel, indefinitely. This has happened twice in production, from two different checks. Gate such checks on the engine in use, or key them on wire bitrate, which exists for both.

System metrics

Measured for the appliance as a whole and shown alongside the channel list, because most capacity problems are visible here before they are visible per channel:

• CPU usage. Aggregate across cores.
• Memory usage. The binding constraint on these appliances. Watch it before adding channels rather than after — see Requirements.
• CPU temperature. Fanless boxes in cupboards behave differently in August.
• Per-interface bitrate, inbound and outbound. Inbound tells you what the uplink is pulling from your origins; outbound tells you what the televisions are being sent, and on which interface, which is frequently the fastest way to notice that it is the wrong one.

DVB-C metrics

Where a DVB-C modulator is fitted, each multiplex reports its running state, its uptime and a restart count. See DVB-C multiplexing.

Health checks Castmux performs

Castmux supervises its own channels. Each condition below is detected continuously and has a fixed, deliberate response.

Condition	What it means	What Castmux does
No data at startup	The engine started but nothing has been produced yet.	Allows a grace period — cold starts legitimately take seconds — then restarts.
Mid-stream silence	A channel that was producing data stopped. Usually the origin dropped it.	Restarts once the stall timeout expires.
Output silent, process alive	The engine is running and believes it is fine, but nothing is on the wire. The worst failure mode, because every internal counter looks healthy.	Restarts after several minutes of confirmed silence.
Output dip	Wire bitrate fell but did not stop. Brief dips are normal at segment boundaries and during origin hiccups.	Tolerated inside a grace window measured from the last known-healthy point. Acting on every dip would restart channels that were about to recover on their own.
Repeated early exits	The engine keeps dying shortly after start. Normally a source that is unreachable or unusable.	Backs off progressively rather than hammering the origin. The backoff resets after a sustained healthy run, so an intermittent source does not accumulate a punitive delay forever.
Destination unroutable	The multicast route does not lead anywhere usable. Checked against the routing table from outside the process, because a socket cannot detect it.	Alerts only. Never restarts. See Troubleshooting.
Duplicate senders	Something else is transmitting to the same multicast group — commonly an older deployment that was disabled but not stopped. Televisions receive both streams interleaved and show pixelation.	Alerts. Restarting would not help, and stopping the wrong sender is not Castmux’s decision to make.
Orphan process	An engine process left behind by a previous supervision cycle.	Reaped.

Restart what a restart can fix, alert on what it cannot

This is the rule the table above encodes, and it is worth stating plainly because it explains why some serious conditions produce no restart at all.

A restart is the correct response to a stuck process, a dead connection to an origin, or an engine that has wedged. It is the wrong response to a network fault. If the multicast route points at an interface with no carrier, restarting the channel produces a fresh process that sends into the same void — and it does so every few minutes, filling the log with restart events that bury the one line naming the real cause. A restart loop against a network fault is worse than the fault, because it destroys the evidence.

So Castmux restarts what a restart can plausibly fix, and for everything else it stops, says what is wrong, and leaves the channel alone.

Exports

OpenTelemetry metrics

Castmux exports its metrics over OpenTelemetry, so the figures above can be sent to whatever collector the site already runs rather than only being visible in the interface.

Series are labelled by channel name

Renaming a channel starts a new series; the old one stops receiving points. If you rename channels routinely, expect a discontinuity in dashboards at that moment. Deletes are handled properly — the series is retired rather than left to go stale — so removing a channel does not leave a permanent flat line in your graphs.

Structured events

Alongside the numeric metrics, Castmux emits structured events for the things that change state. These are what you build a timeline from after an incident:

• engine started, engine exited
• restart initiated
• output silent, output recovered, output degraded
• destination unroutable
• duplicate senders detected
• viewers arrived, viewers departed
• settings changed

A restart on its own says little. A restart preceded by “output silent” and followed by “output recovered” is a source hiccup handled correctly; the same restart preceded by “destination unroutable” is somebody having restarted a channel by hand against a network fault.

Logs

Castmux writes structured JSON to stdout, which a container log collector can ingest without a parsing layer. The same material is available per channel in the interface, buffered to 1000 entries or 24 hours, which is normally enough to diagnose a fault reported that morning without shell access to the appliance.

When opening a support case, the per-channel log for the affected channel is the single most useful attachment. Troubleshooting lists the rest.

Roadmap

The following are committed but not yet released. They are listed here so you can plan around their absence rather than discover it during commissioning.

Prometheus endpoint roadmap

A scrape endpoint alongside the OpenTelemetry export, with a supplied dashboard so a site does not have to build panels from scratch.

Email and webhook alerting roadmap

Delivery of the alert conditions above without an external monitoring stack in between.

Transport-stream analyser roadmap

Built-in analysis reporting continuity-counter errors, PES errors and scrambled packet counts per PID — the measurements that distinguish source-side loss from loss introduced locally.

Restart counter and availability report roadmap

Per-channel restart counts and availability over a period. Today restart frequency can only be recovered by reading the logs, which makes “is this channel getting worse?” harder to answer than it should be.