.circleci
.github
.nx
alembic
frontend
scripts
src
tests
.codacy.yml
.deepsource.toml
.env.example
.gitignore
alembic.ini
CHANGELOG.md
create_user.sql
Dockerfile
INGESTION_GUIDE.md
INTEGRATION_STATUS.md
LICENSE
nx
nx.bat
nx.json
project.json
pyproject.toml
query
README.md
REAL_BACKEND_SETUP.md
renovate.json
requirements-lock.txt
requirements.txt
requirements.vercel.txt
reset_and_setup.py
reset_postgres_password.bat
RESET_POSTGRES_PASSWORD.md
run.bat
setup_db_17.py
setup_db.bat
setup_db.py
vercel.json
Engineering Document Control System
A fully offline, on-premise engineering document-control knowledge base delivered as a Model Context Protocol (MCP) server with a FastAPI backend and web dashboard.
Features
- Multi-format Ingestion: PDF, DWG/DXF, DOCX, XLSX, PPTX, and image files
- OCR with Confidence Gating: PaddleOCR/Tesseract at 300 DPI with 0.75 confidence threshold
- Deterministic + LLM Parsing: Regex-based extraction augmented with local LLM (Llama 3.1 8B / Qwen2.5 7B)
- Hybrid Search: pgvector cosine similarity + SQL full-text search
- Grounded Q&A: Natural-language answers with verifiable citations
- Validation Gateway: Configurable rules engine with JSONB rejection notes
- Audit Trail: Complete logging of all operations with SHA-256 hashes
- Human-in-the-Loop: Review queue for low-confidence OCR regions
Architecture
┌─────────────────────────────────────────────────────────────┐
│ FastAPI Application │
├─────────────────────────────────────────────────────────────┤
│ MCP Server (4 tools) │ REST API │ WebSocket │ SPA │
├─────────────────────────────────────────────────────────────┤
│ Ingestion │ OCR │ Parser │ Embeddings │ Search │
├─────────────────────────────────────────────────────────────┤
│ SQLAlchemy 2.0 Async ORM │
├─────────────────────────────────────────────────────────────┤
│ PostgreSQL 18 + pgvector │ Redis (Celery) │ Ollama │
└─────────────────────────────────────────────────────────────┘Quick Start
Prerequisites
- Python 3.14+
- PostgreSQL 18 with pgvector extension
- Redis (for Celery)
- Ollama (for local LLM)
- Docker (optional, for PostgreSQL)
Installation
- Clone the repository:
git clone https://github.com/zaskiamehrunisha-cloud/document-mcp.git
cd document-mcp- Create virtual environment:
python -m venv venv
source venv/bin/activate # Linux/Mac
# or
venv\Scripts\activate # Windows- Install dependencies:
pip install -r requirements.txt- Configure environment:
cp .env.example .env
# Edit .env with your database credentials- Initialize database:
python -m alembic upgrade head- Start the application:
python src/main.pyDocker Setup (Recommended)
# Start PostgreSQL with pgvector
docker run -d \
--name pgvector \
-e POSTGRES_USER=doccontrol \
-e POSTGRES_PASSWORD=secure_password \
-e POSTGRES_DB=engineering_docs \
-p 5432:5432 \
pgvector/pgvector:0.8.3-pg18
# Start Redis
docker run -d --name redis -p 6379:6379 redis:7-alpine
# Start Ollama
docker run -d --name ollama -p 11434:11434 ollama/ollama
# Pull models
docker exec -it ollama ollama pull llama3.1:8b-instruct-q4_K_M
docker exec -it ollama ollama pull qwen2.5:7b-instruct-q4_K_MProject Structure
├── src/
│ ├── main.py # Application entry point
│ ├── mcp/ # MCP server (4 tools)
│ ├── api/ # FastAPI REST + WebSocket
│ ├── ingestion/ # Document extraction pipeline
│ │ ├── pdf_extractor.py # PDF text extraction
│ │ ├── cad_extractor.py # DWG/DXF extraction
│ │ ├── office_extractor.py # DOCX/XLSX/PPTX extraction
│ │ └── orchestrator.py # Pipeline orchestration
│ ├── ocr/ # OCR engine + confidence gating
│ ├── parser/ # Deterministic + LLM parsing
│ ├── embeddings/ # Chunking + on-device embeddings
│ ├── validation/ # Validation gateway
│ ├── search/ # Hybrid search + Q&A
│ ├── submission/ # Document Controller client
│ ├── llm/ # Local LLM client
│ ├── db/ # SQLAlchemy models + session
│ ├── storage/ # File storage
│ └── config/ # Configuration management
├── alembic/ # Database migrations
├── scripts/ # Operational scripts
├── tests/ # Test suite
├── frontend/ # React web dashboard
├── docker/ # Docker configuration
├── requirements.txt # Python dependencies
├── pyproject.toml # Project configuration
├── .env.example # Environment template
└── README.md # This fileMCP Tools
The system exposes 4 MCP tools over Streamable HTTP:
- ingest_document - Upload and process a document
- validate_document - Run validation rules on a document
- search_knowledge_base - Hybrid search across documents
- submit_to_docon - Forward approved documents to Document Controller
REST API Endpoints
POST /upload- Upload documentGET /upload/status/{job_id}- Check processing statusWS /ws/status/{job_id}- Live status updatesPOST /search- Hybrid searchPOST /ask- Natural-language Q&AGET /documents- List approved documentsGET /documents/{id}- Document detailsGET /review/flagged- Low-confidence regions (admin)PATCH /review/{id}- Mark region reviewed (admin)
Web Dashboard
Three user-facing screens:
- Upload Document - Drag-and-drop upload with discipline tagging
- Ask a Question - Natural-language Q&A with citations
- Approved Documents - Browse validated documents
Configuration
Key environment variables (see .env.example for all):
# Database
DB_URL=postgresql+asyncpg://doccontrol:secure_password@localhost:5432/engineering_docs
# LLM
OLLAMA_HOST=http://localhost:11434
OLLAMA_MODEL=llama3.1:8b-instruct-q4_K_M
# Embeddings
EMBED_MODEL=BAAI/bge-small-en-v1.5
# OCR
OCR_DPI=300
OCR_CONFIDENCE_THRESHOLD=0.75
# Web UI
START_WEB_UI=true
WEB_PORT=8000Document Processing Pipeline
- Upload → SHA-256 hash computed for idempotency
- Format Router → Dispatch to appropriate extractor
- Extraction → Native text + OCR fallback if needed
- Confidence Gate → Sub-0.75 blocks → review queue
- Parsing → Deterministic regex + LLM JSON pass
- Chunking → Parent (1024/128) + Child (256/32)
- Embeddings → On-device BAAI/bge-small-en-v1.5
- Storage → PostgreSQL + pgvector
- Validation → Rule evaluation from database
- Submission → Forward to Document Controller if approved
Technology Stack
- Backend: Python 3.14, FastAPI, SQLAlchemy 2.0 async
- MCP: Official MCP Python SDK (FastMCP)
- Database: PostgreSQL 18 + pgvector 0.8.3
- OCR: PaddleOCR 3.6.x (primary), Tesseract 5 (fallback)
- LLM: Llama 3.1 8B Instruct / Qwen2.5 7B Instruct via Ollama
- Embeddings: sentence-transformers, BAAI/bge-small-en-v1.5
- Queue: Celery + Redis
- Frontend: React + Vite + TypeScript
- Infrastructure: Docker Compose
Hard Constraints
- ✅ Zero external API calls - All inference runs locally
- ✅ SQL-only persistence - No in-memory or key-value stores
- ✅ Offline-first - Models pre-provisioned, no runtime downloads
- ✅ Audit-grade - Every operation logged with hash + model version
Development
Run Tests
pytest tests/ -v --cov=srcCode Quality
ruff check src/
black src/
mypy src/Database Migrations
# Create migration
alembic revision --autogenerate -m "description"
# Apply migrations
alembic upgrade head
# Rollback
alembic downgrade -1License
Proprietary - All rights reserved
Support
For issues and questions, please use the GitHub issue tracker.
Built for engineering document control in EPC/industrial capital projects.