Service
RAG development for production systems
RAG development is the engineering work of turning your documents into retrieval that returns the right passage and generation that stays grounded in it. I am Matin Labkhandagh, and I build and repair RAG systems in Python with LangGraph, LangChain and Neo4j, measured with RAGAS and an LLM-as-a-Judge harness, so retrieval quality becomes a number your team watches rather than a feeling.
When you need this
- Retrieval returns a plausible passage from the wrong document and nobody on the team can say why.
- The answers were fine in the demo, then real users asked questions the chunking strategy never anticipated.
- You swapped to a stronger model hoping hallucinations would drop, and the answers got more fluent but not more faithful.
- Every prompt change is a gamble because there is no golden set and no harness to tell you whether you made things better or worse.
- Token cost per answer keeps climbing because the pipeline fills the context window with weakly related chunks.
What you get
- Retrieval quality report: where the current pipeline loses the right passage (ingestion, chunking, embedding, ranking or prompt) and what to change first.
- Golden set of questions with reference answers and supporting source passages, built with your domain experts and versioned with the code.
- RAGAS evaluation harness wired to the golden set, with faithfulness and relevance scores reported per run.
- Claim-level LLM-as-a-Judge that scores each generated statement against the retrieved sources instead of grading the answer as a whole.
- Reranking stage and post-generation grounding filters integrated into the pipeline.
- RAG integration into your LangGraph orchestration, with state and retry handling for flaky retrieval and provider calls.
- Evaluation tests that run in CI and act as the acceptance criteria for the sprint.
- Docker deployment and documentation covering ingestion, indexing, evaluation and how to add a new document source.
Outcomes
- Retrieval that is measured, not assumed: a golden set and a RAGAS harness that run in CI and fail the build when quality regresses.
- Answers that cite the passage they came from, with grounding filters that drop claims the sources do not support.
- A reranking stage that puts the right chunk at the top, so the generator sees less noise and you pay for fewer tokens.
- A written diagnosis of why the current system fails, so your team understands the fix instead of inheriting a black box.
- A pipeline your engineers can extend, with tests as the acceptance criteria and a runbook for the operational side.
How it works
- 1
20-minute technical call
You walk me through the pipeline, the data and the failures you see. I ask about chunking, embeddings, the retriever, the prompt and how you currently decide whether an answer is good. If a RAG sprint is not the right fix, I say so on the call.
- 2
Written diagnosis
I trace a handful of failing questions end to end and write down where each one goes wrong: the passage was never indexed, it was indexed but not retrieved, it was retrieved but ranked below noise, or it was retrieved and then ignored by the model. The diagnosis names the layer to fix and the metric that will prove it.
- 3
Fixed-scope proposal
Scope, timeline and price in writing, sold as the AI Agent Engineering Sprint. The scope lists the golden set size, the evaluation metrics and the pipeline changes, so acceptance is a test run rather than an opinion.
- 4
Implementation and handover
I build the harness first, then change retrieval, reranking and grounding against it. You get the code, the documentation, the Docker deployment and a runbook, plus a walkthrough with your engineers so the evaluation loop keeps running after I leave.
Proof: what I built
In Jozveh-AI I built the retrieval layer of an eight-agent pipeline that turns lecture audio and source material into cited study documents: Neo4j GraphRAG, retrieval and reranking, and source-cited generation, with RAGAS, a custom claim-level LLM judge and post-generation grounding filters closing the loop. The system is covered by 673 tests across 77 files.
Read the Jozveh-AI case studyThe same project included a controlled model bake-off, several models compared under one evaluation harness. Model choice was not the bottleneck. Retrieval quality, grounding and the evaluation loop moved faithfulness more than swapping models did, which is why I start RAG work at the retriever rather than at the model picker.
Read the Jozveh-AI case studyIn OmidGPT, a multi-provider agentic platform, I built the serving side that retrieval-heavy workloads end up needing: semantic caching, complexity-based model routing and per-token cost accounting, so the cost of each answer is visible and stays inside a budget you set.
Read the OmidGPT case study
Questions about rag development for production systems
- Can you improve an existing RAG system, or do you only build from scratch?
- Most of my RAG work is on existing systems. I start by building a golden set and a harness around what you already have, so the baseline is measured before anything changes. Then I fix the layer the diagnosis points to, usually chunking, retrieval or reranking, and re-run the same harness to show the difference. Rewrites are rare and only proposed when the diagnosis justifies them.
- When is GraphRAG worth it over vector RAG?
- When the questions depend on relationships between entities rather than on similarity to a passage: which sections cite which sources, how a concept connects across documents, what depends on what. If your failures are about missing context that lives two hops away from the retrieved passage, GraphRAG is worth the extra modeling. If they are about wrong passages ranked above right ones, fix reranking first. The GraphRAG development page and the GraphRAG versus vector RAG guide go through the decision in detail.
- Can you evaluate an AI agent we already built?
- Yes, that is the AI / LLM Technical Audit: 5 working days, starting at $400. It covers architecture, retrieval, tool calling, prompts, hallucination, latency, token cost and evaluation strategy, and ends with a prioritized implementation roadmap. For RAG-heavy agents the retrieval review and the evaluation strategy are usually the parts that change the roadmap most.
- What does an evaluation harness include?
- A golden set of questions with reference answers and the source passages that support them; RAGAS faithfulness and relevance scores run against that set; a claim-level LLM-as-a-Judge that scores individual statements against the retrieved sources; and a CI job that runs all of it and fails the build on regression. The harness is versioned with the code, so a prompt change and its effect on the score land in the same commit.
- How long does a RAG sprint take and what does it cost?
- The AI Agent Engineering Sprint starts at $1,200 and runs 7–10 working days. A RAG-focused sprint spends the first days on the golden set and harness, the middle on retrieval, reranking and grounding changes, and the last on deployment, documentation and handover. Systems that also need observability, semantic caching and model routing around the pipeline fit the Prototype → Production AI Sprint instead.