CRAG MultiHop Reasoning Engine Architecture & Performance Benchmark
The modern RAG landscape often stalls on multi-hop questions and ambiguous contexts. The CRAG MultiHop Reasoning Engine tackles these head‑on with a modular, containerized stack that fuses query decomposition, self‑grading retrieval, hybrid dense/sparse search, and real‑time WebSocket progress streaming. This deep‑dive article dissects the system’s architecture, data flows, key features, and production benchmarks—so engineers can see how it delivers high‑precision, self‑healing answers at scale.
The Engineering Challenge: Why CRAG MultiHop Reasoning Engine Was Architected
Traditional single‑step RAG fails when a user’s question demands logical chaining across multiple documents or when the retriever surfaces noisy, ambiguous chunks. The CRAG MultiHop Reasoning Engine was born to solve both pain points:
- Multi‑Hop Decomposition: Complex queries are broken into up to three sequential sub‑queries, each triggering its own retrieval cycle.
- Self‑Grading Retrieval (CRAG): Retrieved context is evaluated by a corrective wrapper that tags it as correct, ambiguous, or incorrect, triggering query expansion or external web fallback when needed.
- Hybrid Dense + BM25 Retrieval: Dense embeddings from
intfloat/multilingual-e5-smallare merged with sparse BM25 results and locally reranked via thejina-reranker-v3cross‑encoder. - Real‑Time Pipeline Visibility: WebSocket streams (port 5151) push live events through Daphne, letting users watch retrieval → grading → reranking → generation unfold step‑by‑step.
These capabilities are delivered via a containerized topology that balances isolation, scalability, and observability—key for production AI services.
Core Architecture & Technical Stack Deep-Dive
Deployment Topology
The system uses Docker Compose to orchestrate five core containers:
┌───────────────────────┐
│ Browser (React UI) │
└───────┬───────────────┘
│ Port 5151
┌───────▼───────────────┐
│ Nginx Proxy │
└─────┬───────┬───────┘
│ │
/api & /ws │ Static
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Daphne (Django) │ │ Nginx HTML/Assets │
│ Port 8051 │ └─────────────────┘
└───────┬─────────┘
│
▼
┌───────────────┐
│ Redis (6351) │◄── ASGI Channel Layer / Task Broker
└───────┬───────┘
│
┌───────▼───────┐ ┌───────────────┐ ┌───────────────┐
│ Celery Worker │ │ ChromaDB │ │ PostgreSQL │
│ │ │ (8002) │ │ (5432) │
└───────────────┘ └───────────────┘ └───────────────┘
Nginx fronts the React UI and proxies /api and /ws to Daphne. Daphne runs Django ASGI, handling HTTP requests and persistent WebSocket connections for real‑time streaming. Redis underpins both the ASGI channel layer and Celery task broker. ChromaDB stores document embeddings for fast vector look‑ups, while PostgreSQL persists pipeline metadata and user sessions. Celery workers asynchronously ingest PDFs, TXT, and URLs, chunk them, generate embeddings, and bulk‑index into ChromaDB.
Tech Stack Snapshot
| Layer | Technology | Rationale |
|---|---|---|
| Frontend | React + Vite | Fast dev server, ESM‑native builds, and reactive UI for streaming events. |
| Backend | Django ASGI / Daphne | Native async support for concurrent retrieval, grading, and WebSocket handling. |
| Task Queue | Celery + Redis | Decouples heavy document processing from real‑time query handling. |
| Vector DB | ChromaDB (8002) | Lightweight, plug‑and‑play vector store with metadata filtering. |
| Embedding Model | intfloat/multilingual-e5-small | Balanced multilingual performance with sub‑second retrieval latency. |
| Reranker | Jina Reranker v3 (Cross‑Encoder) | Local, low‑latency cross‑encoder for hybrid dense+BM25 ranking. |
| LLM | OpenRouter (Qwen 30B) | Cost‑effective, high‑throughput generation with strong reasoning. |
| Streaming | WebSockets (Port 5151) | Bidirectional real‑time pipeline progress events. |
Key Features Breakdown & Practical Benefits
Multi‑Hop Query Decomposition
Up to three sequential hops are generated automatically:
- Hop 1: Extract primary entities and keywords.
- Hop 2: Use Hop‑1 results to formulate a second, more focused query.
- Hop 3: Refine or broaden based on intermediate context.
Virus detection across documents. The pipeline aggregates results, passes them through the CRAG self‑grader, and finally reranks via the cross‑encoder before generation.
Self‑Grading Retrieval (CRAG)
The CRAG wrapper evaluates each retrieved chunk using heuristics and a lightweight classifier, labeling it:
- Correct: High relevance, ready for generation.
- Ambiguous: Low confidence; triggers query expansion.
- Incorrect: Strong negative signal; fallback to external web search.
This self‑healing loop prevents hallucination and ensures the LLM only sees trustworthy context.
Hybrid Dense + BM25 Retrieval & Local Reranking
Dense vectors from intfloat/multilingual-e5-small capture semantic similarity, while BM25 adds lexical precision. Both result sets are merged, then reordered by the local jina-reranker-v3 cross‑encoder—eliminating the need for external API calls and reducing latency.
Real‑Time WebSocket Progress Streaming
Every pipeline stage emits an event (e.g., retrieval_start, grading_complete, rerank_done\d>, generation_stream). The React UI listens via WebSockets (port 5151) and renders a live timeline, giving users transparency and confidence in the answer process.
Asynchronous Document Ingestion
Celery workers handle PDF, TXT, and URL uploads. Documents are chunked, cleaned, and indexed into ChromaDB with metadata tags, ensuring the vector store is always up‑to‑date without blocking the query path.
Real-World Deployment & Production Topology
Production uses Docker Compose with service isolation:
- Nginx Proxy—handles SSL, static assets, and smart routing for API vs. WebSocket traffic.
- Daphne + Django—ASGI server with Channel Layers for WebSocket management and background task coordination.
\ Starter code for Redis and Celery ensures reliable task distribution and pipeline resilience. The ChromaDB container stores embeddings, while PostgreSQL persists pipeline logs, user sessions, and configuration flags. This topology allows each component to scale independently and simplifies maintenance.
Data Flow & How It Works: Step‑by‑Step Workflow
1. **User Query Input** – The React UI accepts natural language questions and optional pipeline toggles (Multi‑Hop, CRAG, Reranking, Dense vs. Sparse).
2. **Multi‑Hop Decomposition** – If enabled, the orchestrator splits the query into sequential hops and queues them for retrieval.
3. **Hybrid Retrieval** – Each hop executes dense (ChromaDB) + BM25 search, merging results.
4