#455
Issues/vxcontrol/pentagi/#455

[Enhancement]:Allow MVP and potential Skill features to access external custom knowledge bases

Openmazhi007opened 22d ago

Target Component

Core Services (Frontend UI/Backend API)

Enhancement Description

Problem Statement:

  • Current Limitation: PentAGI MVP can only use built-in pgvector + Graphiti(Neo4j) memory, with no way to connect to independent external knowledge bases. Skill modules are not supported at this moment.
  • Use Case:
    1. MVP agents need to retrieve private security knowledge bases (internal audit docs, vulnerability checklists, enterprise asset rules) during pentest workflows.
    2. If Skill capabilities are planned for future releases, Skills should also query external knowledge bases to get domain-specific reference data.

Proposed Solution:

  • Feature Description: Add support for MVP (and potential future Skill features) to access external custom knowledge sources. Enable pluggable RAG retrieval, configurable connections to external vector databases or knowledge APIs, not limited to the built-in knowledge store.
  • Expected Benefits: Users can integrate their private security knowledge without importing all data into PentAGI's internal database. This improves flexibility for enterprise custom pentest scenarios.

Example Scenario: During an enterprise penetration test, MVP agent queries an external private vector DB containing internal audit standards and custom vulnerability checklists to guide its assessment. If Skill is implemented later, custom skills can also fetch reference knowledge from this external knowledge base.

Questions:

  1. Is external knowledge access for MVP on the roadmap?
  2. Does the project have plans to implement Skill functionality?
  3. If Skills are supported, will they also get external knowledge access capability?
  4. Are there design docs or proposal branches I can follow or contribute to?

Technical Details

No response

Designs and Mockups

No response

Alternative Solutions

No response

Verification

  • I have checked that this enhancement hasn't been already proposed
  • This enhancement aligns with PentAGI's goal of autonomous penetration testing
  • I have considered the security implications of this enhancement
  • I have provided clear use cases and benefits

Activity

4
XinwangTao22d ago

This would be very useful for enterprise pentest scenarios — teams usually keep audit checklists and asset rules in their own knowledge stores, and forcing them to import everything into PentAGI's built-in memory is a big adoption barrier.

A few suggestions to strengthen the proposal:

  1. Define a stable retrieval interface (e.g. a small KnowledgeProvider contract) so pgvector/Graphiti and external vector DBs can be plugged in uniformly, and future Skills reuse the same interface.
  2. Consider a per-flow config toggle for which knowledge sources an agent may query, since external KBs can contain data with different trust/clearance levels than the sandbox.
  3. Clearly separate retrieval from tool execution in the agent's permission model, so a knowledge lookup doesn't imply sandbox access.

Is there an existing issue/discussion where the team has discussed the Skill roadmap? That would help contributors align on where this fits.

XinwangTao22d ago

This would be very useful for enterprise pentest scenarios — teams usually keep audit checklists and asset rules in their own knowledge stores, and forcing them to import everything into PentAGI's built-in memory is a big adoption barrier.

A few suggestions to strengthen the proposal:

  1. Define a stable retrieval interface (e.g. a small KnowledgeProvider contract) so pgvector/Graphiti and external vector DBs can be plugged in uniformly, and future Skills reuse the same interface.
  2. Consider a per-flow config toggle for which knowledge sources an agent may query, since external KBs can contain data with different trust/clearance levels than the sandbox.
  3. Clearly separate retrieval from tool execution in the agent's permission model, so a knowledge lookup doesn't imply sandbox access.

Is there an existing issue/discussion where the team has discussed the Skill roadmap? That would help contributors align on where this fits.

AlaeddineMessadi15d ago

Most of this already works through the built-in Knowledge Base (/knowledges in the UI, since v2.1.0), with one condition: the documents have to be imported into PentAGI, because it does not query an external store. What you add there goes into the same pgvector collection the agents search. The pentester and installer get search_guide, the coder search_code and the searcher search_answer (tools.go). The searches filter only on doc_type and its subtype, not on flow or user (guide.go), so every flow sees them.

To bulk-load checklists or asset rules, create an API token in the UI and loop over POST /api/v1/knowledge/ (router.go). The default User role has knowledge.create (migration):

curl -X POST "https://<host>:8443/api/v1/knowledge/" \
  -H "Authorization: Bearer $PENTAGI_TOKEN" -H "Content-Type: application/json" \
  -d '{"doc_type":"guide","guide_type":"pentest","question":"Checklist: internal web app authentication","content":"..."}'

A guide needs guide_type, an answer needs answer_type and code needs code_lang (knowledge.go). content can be up to 65536 characters, but only the first EMBEDDING_MAX_TEXT_BYTES (8192 by default) are embedded (knowledge.go), so load one document per checklist section rather than whole files. API tokens are refused while COOKIE_SIGNING_SALT is empty or left at salt (auth_middleware.go).

A live connector that queries an external vector DB or knowledge API without importing the data does not exist today.

XinwangTao15d ago

Thanks for the detailed breakdown — good to know the built-in KB already covers the import path and the API supports bulk loading. The remaining gap is the live external connector: teams that can't mirror data into PentAGI (compliance constraints, already-maintained authoritative stores) still need a query-through adapter. A pluggable KnowledgeProvider interface with an external adapter would close that gap without touching the current import path.