Run Status
Every fleet or workflow run writes one record. mur job status and mur fleet status read that same record through the same derivation, so the two surfaces cannot disagree about what a run is doing.
mur job list # every run MUR has recorded
mur job status <run-id> # one run, in full
mur fleet status <fleet-name> # that fleet's most recent run
State and liveness are different questions
The most common way a status lies is by answering only one of them.
State is what the run last wrote down: running, blocked, done, failed, stopped. It is read from disk, never guessed. The one exception is abandoned, described below; it is derived when you ask, never stored.
Liveness is whether the process behind that state is still there. It is computed at the moment you ask and never persisted:
| Liveness | Means |
|---|---|
alive | the orchestrator process exists and its heartbeat is fresh |
STALLED | the process exists, but the heartbeat has gone quiet |
DEAD | the record says running and the process is gone |
unknown | the record was rebuilt from the channel — the heartbeat cannot be recovered |
- | the run already reached a terminal state; liveness does not apply |
A crashed orchestrator therefore reports running / DEAD the moment you ask — not running / alive until some timeout expires, and not failed while it is actually still working.
Abandoned runs
running / DEAD is the right answer for a few minutes: the process may have died between its last heartbeat and writing its result, and the result can still be reconciled from the channel. Past that, nothing will ever record a terminal state, and printing running forever is a lie. Once a DEAD run's last heartbeat is older than the grace window, every surface prints abandoned instead:
state abandoned
liveness DEAD
note process died without recording a result; the record still says `running` (not a failure verdict — rerun to find out)
- One rule, every surface.
mur job status,mur job list,mur fleet status, themur_job_statusMCP tool and themur monitorwatch all apply the same check, so they cannot disagree about the same run. - Not
failed. MUR does not know what the run did, only that it will not say. Liveness still readsDEAD, and the stored state is left untouched;abandonedis a reporting verdict. - Only
DEADqualifies. ASTALLEDrun still has a live process and may yet move. Anunknownrun (rebuilt record) has no heartbeat to measure. A finished run never qualifies. - The grace window follows the heartbeat. It is 30 stale thresholds:
heartbeat_interval_secs × heartbeat_stale_after_intervals × 30, which is 15 minutes at the defaults below. Slowing the heartbeat widens the window with it, so the two settings cannot drift apart.
The heartbeat
A live run stamps its record on an interval. Two settings in ~/.mur/config.yaml tune it:
runs:
heartbeat_interval_secs: 10 # how often a running orchestrator stamps
heartbeat_stale_after_intervals: 3 # missed intervals before STALLED
Three intervals tolerates one lost tick plus scheduling jitter without calling a healthy run dead. Both values are rejected at load if set to 0 — a zero interval would make every healthy run report STALLED instantly — and fall back to the defaults above.
Rebuilt records say so
A run's record can be re-derived from its channel if the cached copy is lost or corrupt. Everything comes back except the heartbeat, which is the one fact the channel never carried. A rebuilt record admits this rather than guessing:
state running
liveness unknown
pid unknown (record rebuilt)
heartbeat unknown (record was rebuilt from the channel)
It does not print a stale pid as though a process were still there.
An unreadable record is reported as unreadable. A permission error or a corrupt index is an error you get to see, never silently rendered as a run that never existed.
Steps say why they failed
The record carries the run's steps, and a terminal step carries the reason it ended that way.
$ mur fleet status develop-rust
state failed
liveness -
step pm failed pm
└─ delegate failed: no response within the step window
step frontend failed frontend
└─ connect: error sending request for url (http://127.0.0.1:8088/v1/messages)
Without the reason a failed run answers what and never why, and the only route to a cause is grepping agent logs. A step that fails with nothing to say says that too — no output; exit code 1 beats an empty field. A done step carries no reason, and a retry clears the previous attempt's rather than leaving it beside a running state.
Fleet status
mur fleet status <name> is a lookup plus the shared renderer, not a second status computation. It finds the most recent run whose record names the fleet's channel and hands it to the same code mur job status uses.
$ mur fleet status builder
run run-4f2a…
kind Fleet
label fleet run
state running
liveness alive
pid 48213
started 2026-08-17T02:14:09+00:00
heartbeat 2026-08-17T02:41:52+00:00
channel fleet-builder
If a neighbouring run's index is corrupt it is skipped with a warning, not silently — one bad record must never hide the fleet's real run, and it must never disappear without saying so either.
See also
- Fleet Loops — when a looping fleet decides it is done
- Settlement — what a single turn actually verified