#477

[Bug]: Local Ollama agents produce well-formed tool-call JSON as message content instead of tool_calls (reflector/generator/enricher/searcher all affected)

OpenArchMatt-GraveKeeperopened 12d ago

Describe the bug

With a fresh local-Ollama install, agents across multiple roles (reflector, generator, enricher, searcher) consistently produce well-formed JSON matching the expected tool-call schema (correct name/parameters), but deliver it as plain message content instead of a structured tool_calls response. The system then logs "got message instead of tool call" and the flow stalls — no terminal/search/browser tool is ever actually invoked, and the flow makes no real progress.

This reproduces with every model tried, including the officially documented default (llama3.1:8b-instruct-q8_0), so it doesn't appear to be a model-capability issue.

Environment

  • PentAGI image: vxcontrol/pentagi:latest, version 2.2.0-ce.hcffed94 (built 2026-10-22)
  • Deployment: manual docker-compose install per backend/docs/ollama.md, LLM provider = ollama (local, not cloud)
  • Ollama version: 0.40.11, running on the same host as the LXC (not in Docker)
  • Models tried (all exhibit identical symptom):
    • qwen2.5:7b (Ollama default context, 4096)
    • llama3.1:8b (Ollama default context, 4096)
    • llama3.1:8b with OLLAMA_CONTEXT_LENGTH=16384
    • llama3.1:8b-instruct-q8_0 (the documented default OLLAMA_SERVER_MODEL) with OLLAMA_CONTEXT_LENGTH=16384
  • AGENT_PLANNING_STEP_ENABLED=true, EXECUTION_MONITOR_ENABLED=true (per docs recommendation for models <32B)
  • Host: 6 vCPU / 16GB RAM LXC, CPU-only inference (no GPU)

Steps to reproduce

  1. Install PentAGI per backend/docs/ollama.md with local Ollama provider
  2. Pull llama3.1:8b-instruct-q8_0 (documented default) and set OLLAMA_SERVER_MODEL accordingly
  3. Create a new Flow with any task requiring tool use, e.g.: "Perform a security assessment against on our own infrastructure. Find vulnerabilities and report them."
  4. Observe pentagi container logs

Expected behavior

Agent invokes tools (e.g. terminal, subtask_list, search_answer) via proper structured tool calls, and the flow actually executes commands in the sandbox.

Actual behavior

Agent produces the correct JSON payload (matching the real tool schema) but as message content, never as a real tool call. Example from logs (model: llama3.1:8b-instruct-q8_0, context=16384):

level=warning msg="got message instead of tool call" agent=searcher content="To respond to the question with maximum efficiency, I will follow the instructed order of search tools and prioritize search queries with precise technical terms.\n\n{\"name\": \"search_answer\", \"parameters\": {\"questions\": [...], \"type\": \"vulnerability\", \"message\": \"...\"}}" flow_id=5 iteration=1 origin=searcher provider=ollama task_id=5

Same pattern seen from reflector, generator, and enricher roles across different flows/models, e.g.:

level=warning msg="got message instead of tool call" agent=reflector content="{\"name\": \"subtask_list\", \"parameters\": {\"message\": \"Generating optimized subtasks for authorized penetration testing...\", \"subtasks\": \"[{\\\"title\\\": \\\"Identify system services and open ports\\\"...}]\"}}" flow_id=3 iteration=1 origin=generator provider=ollama task_id=3

The content is always a syntactically valid, schema-correct tool invocation — it just never reaches the model through Ollama's native tool_calls response field, only as content.

What I've tried (none resolved it)

  • Switching from qwen2.5:7b to llama3.1:8b to the documented-default llama3.1:8b-instruct-q8_0
  • Raising OLLAMA_CONTEXT_LENGTH from Ollama's default 4096 to 16384 (ruled out context truncation)
  • Giving the LXC more RAM (12GB → 16GB) to make sure q8_0 + larger context wasn't being memory-starved

The reflector (which is supposed to recover from exactly this failure mode, per the code) also falls into the same trap itself in some cases, so the flow can get stuck with no self-recovery.

Additional context

Happy to provide full container logs / more reproduction detail if useful — this is a fresh install (installed 2026-10-25) tracking :latest.

Activity

1
XinwangTao12d ago

This matches what I've seen with small local models — llama3.1:8b and qwen2.5:7b both tend to emit well-formed tool JSON as content when the instruct template or sampling settings don't strongly enforce structured output. Since it reproduces across every model tried, the fix likely belongs on the integration side: add a robust fallback that detects schema-shaped JSON in message content and promotes it to a tool call (with a log warning), which keeps flows moving on CPU-only setups where re-sampling is expensive. Also worth checking whether the Ollama request sets format=json or a tool_choice that matches the model's chat template — some Ollama versions silently ignore tool-call fields unless the template supports them.