#1632

[BUG] Staging API specifications drops local external $ref files

Opensamirhvbropened 9d ago

Describe the bug A local OpenAPI document can pass target detection and scope extraction but lose part of its contract during sandbox staging. stage_api_specs() copies only the main file, so a relative $ref to a sibling schema remains in that file while its destination is absent from /workspace/api-specs.

This affects ordinary split specifications, such as openapi.json plus schemas/User.json: the agent is instructed to read the full request schema but cannot obtain the referenced required fields from the staged contract.

To Reproduce On the commit below, after installing frozen dependencies, run this offline setup-only reproducer. All paths are temporary, the reference stays inside the spec's parent directory, and no network target is contacted:

LITELLM_MODE=PRODUCTION LITELLM_LOCAL_MODEL_COST_MAP=True STRIX_TELEMETRY=0 uv run --frozen python - <<'PY'
import argparse
import json
import tempfile
from pathlib import Path
from unittest.mock import patch

from strix.interface.scan_setup import build_targets_info
from strix.interface.utils import stage_api_specs

with tempfile.TemporaryDirectory() as temporary:
    root = Path(temporary)
    source = root / "source"
    (source / "schemas").mkdir(parents=True)
    (source / "schemas" / "User.json").write_text(json.dumps({
        "type": "object", "required": ["name"],
        "properties": {"name": {"type": "string"}},
    }))
    reference = "./schemas/User.json"
    spec = {
        "openapi": "3.0.3", "info": {"title": "Offline fixture", "version": "1"},
        "servers": [{"url": "https://api.example.test"}],
        "paths": {"/users": {"post": {
            "requestBody": {"required": True, "content": {
                "application/json": {"schema": {"$ref": reference}},
            }},
            "responses": {"201": {"description": "Created"}},
        }}},
    }
    path = source / "openapi.json"
    path.write_text(json.dumps(spec))
    with patch("tempfile.gettempdir", return_value=str(root / "staging")):
        args = argparse.Namespace(target=[str(path)], target_list=None)
        build_targets_info(args)
        sources = stage_api_specs(args.targets_info, "relative-ref-repro")
        staged = Path(sources[0]["source_path"])
        print("source reference exists:", (source / reference).is_file())
        print("staged reference exists:", (staged / reference).is_file())
        print("staged files:", sorted(p.name for p in staged.iterdir()))
PY

Actual output:

source reference exists: True
staged reference exists: False
staged files: ['openapi.json']

Expected behavior Either make supported local references available with the same relative paths, or reject an unsupported multi-file contract before the run with an actionable instruction to bundle it. Silently accepting the main file leaves the sandbox with an incomplete contract.

Additional context / proposed decision

  • Staging copies one file per target, while the API-spec skill asks the agent to resolve schemas.
  • A bounded first option is support for local references contained within the spec directory, preserving relative paths and explicitly handling cycles/missing files. Another is a documented bundled-spec requirement with validation. Which contract is intended?
  • References outside that directory, symlinks and remote URLs require explicit policy; this report does not propose copying an arbitrary directory tree or automatically fetching remote references.
  • This is distinct from #1619 / #1621, which restore already-staged files on resume: this case loses a referenced file during the initial staging itself.
  • I checked the open/closed PR and issue inventory and #866's discussion. The older $ref finding there concerned fields dropped by host-side normalization; that parser was removed. This reproduction concerns an external file that never reaches the sandbox.

System Information:

  • OS: macOS 27.0.1, arm64
  • Strix commit: 6b7ae2ebe4d47037a927aade23265ff4e4f27e84 (current main)
  • Python: 3.12.14; frozen uv.lock dependencies; isolated temporary HOME
  • LLM: none. No Docker sandbox, target scan, or live Postman request was used.

The existing tests/test_api_spec.py and tests/test_api_spec_targets.py suites pass (39 tests). These reproductions exercise setup/staging directly; they do not measure end-to-end model behavior.

Activity

1
ToniAdreal7d ago

Confirmed the mechanism at current main (6b7ae2e, the same commit the report pins):

  • stage_api_specs (strix/interface/utils.py:1517-1555) copies exactly one file per target with a single shutil.copy2(source, staging / name). There is no $ref walk anywhere in utils.py (grep for $ref: zero hits), so a sibling file like schemas/User.json is never staged.
  • The function's own docstring states the assumption outright: "A spec is a single file the agent reads, not a tree it works in." That assumption is what breaks here, because the api-spec skill (strix/skills/custom/api_spec_testing.md) instructs the agent to build its inventory by resolving $ref/allOf request-body schemas. The staged contract and the instructed task disagree for any split spec.
  • The failure is silent in a specific way worth naming: target detection and scope extraction pass, because they read the host-side file. Only the sandbox copy is incomplete, so the agent sees a dangling $ref with no signal for whether the referenced schema is intentionally absent or was lost in staging.

On the fix shape, I agree with the reporter's bounded framing. Two options that keep the current single-directory staging model intact:

  1. Copy local references contained within the spec's directory, preserving relative paths, with explicit handling for cycles and missing files (fail setup loudly in those cases rather than staging a dangling ref).
  2. As the guard for everything outside that bound (out-of-directory refs, symlinks, remote URLs): reject the multi-file contract at setup time with an actionable "bundle your spec" message, instead of staging the main file alone.

Option 1 for in-directory refs plus option 2 as the guard seems like the coherent pair: split specs like openapi.json + schemas/User.json are ordinary, not exotic, so silently accepting only the main file is the worst of the three outcomes. This is distinct from #1619/#1621 (those restore already-staged files on resume); this loss happens at initial staging.

Scope honesty: this is source-level confirmation at the pinned main tip, not a runtime rerun of the reporter's offline reproducer (whose output already demonstrates the staged-files list precisely).