#434
[Enhancement]: Add Webz.io News Search as a search engine
Target Component
External Integrations (LLM/Search APIs)
Enhancement Description
Problem statement:
- Current limitation: PentAGI can search the web through Tavily, Perplexity, Firecrawl, and SearXNG, but it has no global news index with publication dates, language, country, and source filters.
- Use case: research and monitoring flows that need recent news articles, not only general web pages.
Proposed solution:
- Add a Webz.io News Search searcher beside the existing engines.
- Webz.io News Search is a global news index queried in natural language, with filters for date, language, country, domain, sentiment, and category. It is also available as a hosted MCP server at https://news-search-mcp.webz.io/mcp. The tool name is
news_search_by_webz. The user suppliesWEBZ_API_TOKEN. - Docs: https://docs.webz.io/docs/webz/news-search-api-mcp
I work on Webz.io News Search.
Technical Details
Follow backend/pkg/tools/searchers/tavily.go. Implement Searcher from searchers.go: Engine(), IsAvailable(), and Handle(). Read WEBZ_API_TOKEN from config, call the hosted MCP tool news_search_by_webz, and return the same result string shape the orchestrator already consumes. Add a searcher test next to tavily_test.go.
Alternative Solutions
Users can already attach the hosted MCP server themselves. A native searcher would put Webz in the same engine list and fallback order as Tavily, without a separate MCP setup.
Verification
- I have checked that this enhancement hasn't been already proposed
- This enhancement aligns with PentAGI's goal of autonomous penetration testing
- I have considered the security implications of this enhancement
- I have provided clear use cases and benefits
Nice addition — a news-search engine fills a real gap for OSINT workflows, and MCP-based discovery keeps it decoupled from the core searcher list. Two practical questions: (1) is the plan to register it as a normal searcher provider (so it participates in existing per-agent searcher assignment), or as a dedicated MCP tool with its own agent exposure? (2) Webz.io requires an API token (WEBZ_API_TOKEN) — will it reuse the existing per-provider credential storage pattern from the other search providers, or does it expect an env var like the current LLM/search providers? If the latter, a Settings → Providers entry (similar to what #398 proposes for LLM providers) would make it usable for self-hosted users without touching .env.