AI & Machine Learning · Open-source build
Grounded assistant with an evaluation harness
Retrieval-augmented answers over a document corpus, with citations the reader can check and a test set that decides whether a change ships.
What it is
Documents are chunked, embedded and stored in PostgreSQL with pgvector. Hybrid search combines vector similarity with keyword ranking, and a reranker orders the candidates before the model answers with inline citations. An evaluation set of question, expected sources and reference answer runs on every change; answer quality, groundedness and citation precision are scored and trended.
What the build consists of
- 01Ingestion pipeline with chunking tuned per document type
- 02pgvector storage with hybrid search and a cross-encoder reranker
- 03Answer generation with enforced citations and refusal when evidence is thin
- 04Evaluation harness scoring faithfulness, relevance and citation precision
- 05Prompt and retrieval changes gated by the evaluation, not by opinion
- 06Feedback capture that feeds new cases into the test set
What you see in a demo
Asking a question the corpus cannot answer and getting a refusal, then asking one it can and following every citation to its source. Then a retrieval change that the evaluation fails, and why.