Agent Vision
An agent's tools can hand back images, not just text. Point read_file at a screenshot and a vision-capable agent looks at it.
mur agent perm allow-read car-eye ~/.mur/artifacts
mur agent restart car-eye # entitlements load at startup
Then ask it to read a picture:
> read ~/.mur/artifacts/dashboard.png and tell me which warning lights are on
What reaches the model
| Source | Result |
|---|---|
read_file on PNG, JPEG, GIF, WebP | image input — the model sees it |
| An MCP server returning `` content | image input |
read_file on SVG | text — it is markup, and reading it is the useful answer |
| Anything else | text, as before |
The tool also returns a short line of text alongside the picture ([image /path — image/png, 91820 bytes]), so a transcript still reads sensibly and a model without vision is not left with nothing.
The failure this replaces
Before this, read_file decoded a JPEG as lossy UTF-8 and handed the model mojibake. A model has no way to signal that it never received a picture, so it answered from the filename and the surrounding prompt instead. In one observed run an agent reported a car's make, model and year with confidence: 0.85 — for a photo it had never been shown.
Changing the model does not fix that. It changes whether the guess is honest, not whether the image arrived.
Limits
- The model must support vision. A text-only model rejects image blocks; pick a vision-capable entry in
~/.mur/models.yaml. - One image per file read, up to 3.75 MB decoded. A larger file returns an error telling you to resize it — never a truncated one, because a truncated image is a corrupt file.
- Unsupported types are dropped, not sent. A provider rejects an entire turn over one bad image, so losing the picture beats losing the turn.
- OpenAI-backed agents cannot see tool images. The OpenAI
role: "tool"message takes a plain string and the protocol has no slot for an image; those agents keep the text line. Use the user-turn path below instead.
Sending an image from outside
To hand an agent a picture directly — from a script, a webhook, another agent — attach it to the message rather than making the agent fetch it:
mur agent send car-eye "$(jq -nc --arg b64 "$(base64 -i photo.jpg | tr -d '\n')" ',
}
]
}')"
data must be an object with a base64 key, not a bare string. One image per message — the first image/* part is the one that reaches the model.