Production RAG Platform Architecture
Requirements
Start with:
- Corpus size and update frequency.
- Query volume and latency target.
- Data sensitivity and tenant boundaries.
- Answer quality and citation requirements.
- Availability and recovery objectives.
- Budget and operational constraints.
Architecture
The core flow is:
Documents → ingestion → chunks/embeddings → index → query → retrieval/ranking → context → model → response
Keep ingestion and serving paths independently observable and scalable.
Security
Authorization must be applied to the information a user is allowed to retrieve, not only to the API endpoint. Protect source documents, indexes, credentials, model access, and audit records.
Cost and scaling
Measure cost per query and per indexed document. Separate expensive asynchronous ingestion work from latency-sensitive serving. Establish budgets before optimizing individual components.
Failure modes
Plan for stale indexes, weak retrieval, model errors, quotas, dependency outages, and authorization mistakes. Each failure should have a detection signal, mitigation, and verification step.
Connected knowledge
RAG? → datasets? → feature engineering → architecture → interview scenario → quiz.