Robotics AI Tutor

The problem

Students asked precise questions ("how do I tune a PID controller for this servo?") and got vague, generic answers. The manuals had the answer. The model wasn't reading them properly.

The first version was slow too. Every answer took more than four seconds, and most of that time went to retrieval, not the LLM.

The approach

1. 01

   Smaller, overlapping chunks

   Dropped from 1024-token chunks to 512 with a 50-token overlap. Less irrelevant text in the context window meant the model stopped blending two sections into one wrong answer.

2. 02

   Hybrid retrieval

   Pure vector search kept missing exact technical terms like "PID controller" or part numbers. Adding keyword scoring (BM25) next to embeddings fixed it.

3. 03

   Embedding cache

   Students ask the same questions in slightly different words. Caching query embeddings and hot retrievals took repeat questions under a second.

4. 04

   Streaming with sources

   Answers stream token by token over FastAPI, and every answer ends with the document and section it came from.

The outcome

Response time dropped from 4.2s to 1.2s, and context accuracy went from 71% to 89% on our internal question set.

The main lesson carried into every later project: in most slow RAG systems the problem is retrieval, not the model.