#22
Add image support to readFile via toModelOutput
Add image support to readFile via toModelOutput
Summary
readFile currently treats image files like text, which means models receive
garbled output instead of usable visual input.
This change adds image support to readFile by detecting common image types,
reading them as binary when the sandbox supports readFileBuffer, and returning
them through toModelOutput as image-data content parts.
Changes
- Add optional
readFileBuffer?to theSandboxinterface - Wire
readFileBufferthrough the just-bash and Vercel sandbox adapters - Detect supported image extensions in
readFile - Return image results as base64 plus
mediaType - Convert image tool results to
image-dataviatoModelOutput - Add unit coverage for image output, model output conversion, and fallback behavior
- Update README and AGENTS docs to match the new behavior
Supported image types
- png
- jpg
- jpeg
- gif
- webp
Compatibility
- Text file behavior is unchanged
readFileBufferis optional, so existing custom sandboxes continue to work- If
readFileBufferis not implemented, image files fall back to the existing text path - No new dependencies are added
Testing
- Added unit tests for image reads returning
{ data, mediaType } - Added unit tests for
toModelOutputimage conversion - Added fallback coverage for sandboxes without
readFileBuffer pnpm validatepasses