InsightRAG
Answers that know when they're not sure.
A retrieval-augmented document assistant: upload PDFs, ask questions in plain language, and get answers grounded in the documents — each one returned with a confidence score so people know how far to trust it.
- Attribution
- Independent Project
- Period
- 2025
- Category
- AI & Automation
3-signal
Confidence scoring
01The challenge
Standard RAG chatbots answer every question with the same confidence, even when the retrieved evidence is weak or irrelevant. In compliance-critical workflows, that kind of confident hallucination is unacceptable.
02The solution
A LangGraph agent that rewrites and classifies each question, retrieves from the uploaded documents, grades the evidence, refines the query when it falls short, and attaches a calibrated confidence score to every answer.
03Key features
- 01
PDF upload, parsing and chunking
- 02
Semantic search with HuggingFace embeddings and FAISS
- 03
Query rewriting and question classification before retrieval
- 04
Evidence grading — relevant, needs refinement or irrelevant — with refinement loops
- 05
A confidence score and label on every answer
- 06
Conversation memory across turns
04Engineering approach
How it's built.
- FastAPI
- LangGraph
- LangChain
- FAISS
- Groq
- HuggingFace embeddings
- PyPDF
- 01
Confidence blends three signals: vector similarity from FAISS distances, lexical overlap between the question and the retrieved context, and the LLM grader's verdict — scaled conservatively to avoid inflated scores.
- 02
When evidence is insufficient, the agent refines the query and retries; answers that remain low-confidence are surfaced as such so they can be routed to human review instead of being trusted blindly.
- 03
A modular FastAPI structure — routers, services, models and utilities — built to extend to new document types and retrieval strategies.
05Project gallery
Inside the system.
Architecture views drawn from the project's documented design and its public source code.
System view · architecture from verified project notes
Grade the evidence before answering.
Every question passes through rewriting, retrieval and grading before an answer — and a confidence score — is produced.
- 0102
Upload PDF
Parsed, chunked and embedded
- 0203
Rewrite & classify
Sharpens the question for retrieval
- 0304
Retrieve
Top matches from the FAISS index
- 0405
Grade evidence
Relevant · needs refinement · irrelevant
- 0506
Refine or answer
Loops back when evidence falls short
- 06
Confidence score
Vector + lexical + grader blend
Key design decisions
- Three independent signals feed the confidence score, not the LLM's own opinion alone.
- Weak evidence triggers query refinement instead of a guess.
- Low-confidence answers are labelled so people can review them.
backend/app/
├── main.py FastAPI entrypoint
├── config.py configuration
├── models/schemas.py request & response models
├── routers/api.py API routes
├── services/rag_service.py LangGraph RAG pipeline
├── services/memory_service.py conversation memory
└── utils/pdf_utils.py PDF text extraction06Outcome
What it delivered.
A working document Q&A agent that grades its own evidence and makes its uncertainty visible, rather than presenting every answer as equally reliable.
- Confidence scoring: vector + lexical + LLM grader
- 3-signal
As published in the founder's portfolio. No figures on this page are estimates or projections.
07Project attribution
Independent ProjectAn independent project designed and built by our founder, Arslan Arshad. It is a technical demonstration, not a client engagement.
Sources: Founder portfolio (opens in a new tab) · Repository README & source (opens in a new tab)