RAG system from scratch
Full pipeline: ingestion and normalization, chunking strategy chosen from the corpus’s real structure, embeddings, vector store, hybrid search with reranking, grounded generation and evaluation.
4–8 weeks
RAG systems
RAG that answers well — and knows when to stay quiet
Retrieval-Augmented Generation connects a language model to your company’s real information. Done right, it answers with verifiable sources. Done wrong, it answers confidently about what it does not know.
Technical capabilities
IngestionChunkingEmbeddingsHybrid searchRerankingMetadata filteringRAGASCitations
2M+users served by the RAG system we ran in production
3 yearssustaining it — which is where the real failures appear
10countries with different catalogs and languages
Key takeaways
- RAG — retrieval-augmented generation — inverts the order: it searches the company's own information first and only then writes, citing the source of each claim.
- The most frequent RAG failure is answering correctly about the wrong entity, because similar entities sit almost on top of each other in vector space. It is fixed with structured metadata at ingestion and pre-ranking filtering.
- Quarl measures retrieval and generation separately: recall@k, MRR and NDCG on one side; groundedness, faithfulness and relevance on the other, with RAGAS and LLM-as-judge.
- The track record is a RAG system operated for three years over a catalogue with high entity similarity, serving two million users across ten countries.
Operating track record
FEMSA loyalty platform
Three years operating the system in production — not one delivery and an exit.
Cross-entity contamination
The most frequent failure in RAG architectures, solved with structured metadata at ingestion.
Fixed scope and date
The proposal arrives within 48 hours with fixed scope, price and date. If scope changes, it is quoted separately and approved first.
What RAG is
Search first, answer second
A language model knows everything except your company: not your prices, your inventory, your policies or an order’s status. RAG solves that by inverting the order: it searches your information first, then writes the answer using only what it found, and cites where it came from.
The idea is simple. The execution is everything. A mediocre RAG and a good one look identical in the demo and separate brutally by month three.
The five failures
What we almost always find in a RAG system that does not work
If any of these sound familiar, your problem is fixable and probably cheaper than you fear.
- Right answer, wrong entity. The most common failure and the most dangerous, because the answer sounds correct and nobody audits it. It happens because "Premium plan pricing" and "Basic plan pricing" are nearly identical in vector space. Fixed by extracting structured metadata at ingestion and filtering by entity before ranking.
- It invents when it does not know. A model prefers a plausible answer to admitting ignorance. Corrected with grounding constraints, mandatory citation of the source passage, and explicit refusal when retrieval comes back empty.
- Nobody knows whether it answers well. If your vendor cannot show a quality number, it is not that the number is bad: it was never measured. Solved with a reference set and systematic measurement.
- The bill grows faster than usage. Because every question gets the most expensive model with all available context, just in case.
- The information is stale and nobody noticed. Loading was manual, done once, and nobody defined how it gets refreshed.
Services
What we do with RAG
Diagnostic of an existing RAG
Reference set built from real questions, retrieval and generation measured separately, and failures ranked by impact with their fix.
1 week
Retrieval optimization
When the system exists but retrieves badly: query rewriting, metadata filtering, reranking, chunking strategy adjustment.
3–5 weeks
Evaluation layer
Reference set, retrieval and generation metrics, A/B testing between configurations and tracing. So your team can change things without breaking them.
2–3 weeks
How it is measured
The numbers we require before saying it works
| Dimension | Metric |
|---|---|
| Retrieval | recall@k |
| Retrieval | MRR |
| Retrieval | NDCG |
| Generation | Groundedness |
| Generation | Faithfulness |
| Generation | Answer relevance |
| Operations | Cost per query |
| Operations | p95 latency |
We work with RAGAS and LLM-as-judge alongside the classic retrieval metrics.
Where this comes from
This is not theory. We operated a RAG system for three years over a knowledge base of product, points, promotions and policies, with high similarity between entities, serving more than two million users across ten countries.
The cross-entity contamination failure — number one on the list above — was solved there, with structured metadata at ingestion and pre-ranking filtering. It is the kind of problem that only appears once the system has been in production for months and the catalog has grown.
Stack
What we build RAG with
LangChainLlamaIndexAzure AI SearchPineconepgvectorQdrantOpenAIAnthropic ClaudeCohere RerankBM25RAGASLangSmithPostgreSQLPythonTypeScript
Related services
Language models
LLM applications with structured output and tool calling.
View serviceAI agents
LangGraph orchestration, durable state and human approval on steps with consequences.
View serviceChatbots and assistants
With the evaluation set run before it ever reaches production.
View serviceProject rescue
The system is already in production and answers badly. We measure it and determine what to fix.
View serviceFrequently asked questions
What exactly is RAG and why is ChatGPT not enough?
RAG stands for Retrieval-Augmented Generation. Instead of asking the model to answer from memory, the system searches your information first, selects the relevant passages and asks the model to write using only those. ChatGPT on its own does not know your prices or policies: ask it and it answers generically or invents. With RAG the answer comes from your documentation and can be verified because it cites the source.
Is RAG better than fine-tuning on our data?
Almost always RAG, for a practical reason: your information changes. Fine-tuning teaches a model to speak a certain way, not to memorize updatable facts — when prices change you would have to retrain. RAG separates knowledge from the model, so updating means reindexing. Fine-tuning makes sense for tone, format or very specific domains, and sometimes both are combined. We tell you which applies on the first call.
How much information do we need for this to be worth it?
There is no strict minimum, but below roughly twenty FAQs with information that barely changes, RAG is over-engineering: a subscription platform solves that more cheaply. RAG starts paying off when the corpus is large, changes often, has many similar entities, or when a wrong answer carries real cost.
What document formats do you support?
PDF, Word, Excel, HTML, Markdown, plain text, web pages and databases. Also scanned documents through OCR. What takes work is not the format but the structure: tables, documents with many sections and catalogs with variants need different chunking strategies, and that gets decided by looking at your actual corpus, not by default.
How do you prevent it from inventing answers?
Four things combined: grounding constraints in the system prompt, mandatory citation of the passage backing each claim, explicit refusal when search returns nothing relevant, and groundedness measurement over a reference set to catch when the system starts drifting from sources. None of the four is sufficient alone.
How often does the index need updating?
It depends on how fast your information changes. For catalogs and pricing we usually schedule daily or event-driven reindexing; for policies and documentation, weekly. What matters is that ingestion is reproducible and has alerts when a source has gone too long without refreshing, because the most common failure mode is not breaking: it is going stale without anyone noticing.
Does our information leave our infrastructure?
Only if you want it to. We can deploy on Azure OpenAI or Amazon Bedrock inside your own subscription with the vector store in your cloud, so information never leaves your perimeter. In any configuration we use enterprise plans where API data is not used for training, and we filter personal data before sending.
What does running a RAG system cost per month?
Model consumption plus the vector store. For a mid-market company it usually lands between USD 150 and 900 per month depending on query volume and corpus size. Model routing — a cheap one to classify and summarize, a capable one for the final answer — and context trimming exist precisely to keep that figure predictable as usage grows.
Book 15 minutes
Tell us what you are building, or what stopped working. You leave the call with a concrete answer: it can be fixed, it can be built, or it isn't worth it.