#1637

Proposal: Outcrawl as an optional web_search backend

Openheydryftopened 7d ago

Strix's web_search tool currently uses Exa or Perplexity. I'd like to propose Outcrawl (https://outcrawl.ai/docs/api/overview) as an optional third backend, selected the same way when its key is set (OUTCRAWL_API_KEY).

Outcrawl's search endpoint (POST https://api.outcrawl.ai/v1/search) returns title, URL and snippet per hit, and can optionally render each hit to Markdown in the same call, so the tool could return page content for CVE advisories and vendor docs without a second fetch.

Scope I have in mind: one function in strix/tools/web_search/tool.py next to the Exa path, the settings key, tests mocking the HTTP call, and a docs line. No new dependency (requests is already used).

Disclosure: I work on Outcrawl, and this proposal and the follow-up change are prepared with AI assistance. Happy to open the PR if this is something you'd accept, or drop it if you'd rather keep the backend list as is.

Activity

1
LuciyVI7d ago

Could SearXNG also be considered as an optional web_search backend?

SearXNG is an open-source, self-hostable metasearch engine that aggregates results from multiple search services. It would be useful for users running local models who also want control over their search infrastructure.

A minimal integration could include:

  • A configurable SEARXNG_BASE_URL pointing to a user-managed instance.
  • Requests to /search with q and format=json.
  • Mapping result titles, URLs, and content snippets into Strix’s existing search response format.

searx.space lists public instances, but these should remain an optional, user-selected choice: JSON access may be disabled, and rate limits or restrictions on automated requests may apply. A self-hosted instance would be the recommended setup.

JSON output must be enabled in the instance’s search.formats configuration; see the Search API documentation.

This would complement the Outcrawl proposal by adding a self-hosted search option. Would maintainers be open to supporting it?