Naive RAG is a lie: How we actually parse 10M lines of code

Most AI code tools are just fancy wrappers around cat **/*.py | llm. That works for a Hello World script. It fails the second you introduce a complex dependency graph or a deeply nested monorepo.

At git11, we stopped pretending that a simple vector database is enough. You can't just chunk code into 512-token blobs and hope the semantic search finds the right interface definition. It doesn't.

Code isn't prose. It's a graph. If you treat UserAuth service = new UserAuth(); as just a string of text, you've already lost. Here is how we actually build a mental model of your repository.

The failure of chunking

Standard RAG (Retrieval-Augmented Generation) breaks code into chunks based on character counts. This is stupid.

If a chunk ends in the middle of a function, the model loses the context of the return type. If the chunking algorithm ignores the imports at the top of the file, the model has no idea what client.execute() refers to.

We don't use fixed-size chunks. We use Tree-sitter.

Every file that enters the git11 pipeline is parsed into an Abstract Syntax Tree (AST). We chunk by nodes, not by lines. A chunk is a class, a method, or a standalone block.

Keeping the logic intact means the embeddings reflect what the code does, not just the words it uses.

Dependency-aware indexing

Searching for getUser in a large codebase returns 50 results. Your LLM shouldn't have to guess which one matters for the current ticket.

We built a custom indexer that tracks references. When we ingest a repository, we don't just store the code. We map the relationships.

If File A calls a method in File B, we inject that metadata into the vector. We call this "contextual enrichment."

When you ask git11 about a bug in the payment flow, our engine knows that StripeWrapper.ts is a dependency of CheckoutService.ts. It pulls both, even if the keyword match for the wrapper is weak.

The Context Window Problem

Everyone is bragging about million-token context windows. They are mostly lying about the performance.

If you dump 100 files into a context window, the model gets confused. It hallucinates variable names from one file into the logic of another. We call this "context poisoning."

We use a two-pass ranking system. Pass one is a fast BM25 scout to find keywords. Pass two is a cross-encoder that re-ranks the top 50 results based on actual structural relevance.

We only send the top 5-7 most relevant snippets to the LLM. Quality over quantity is the only way to get code that actually compiles.

Dealing with Python and TypeScript type hell

Dynamic languages are a nightmare for AI. If the code says data.save(), and data is typed as any, the LLM is guessing.

We run a light static analysis pass during ingestion. We use Pyright for Python and the TypeScript compiler API for TS.

If we can resolve a type, we attach it as a comment to the code snippet before it hits the LLM. It turns out models are much smarter when you tell them exactly what the data structure looks like.

Why we hate vector-only search

Vector databases are great for finding "things that look like this." They are terrible at finding "exactly this function."

If you search for onUpdate, a vector search might return onDelete because they appear in similar contexts. That's a halluncination trigger.

We use a hybrid approach: Qdrant for semantic similarity and a dedicated Postgres index for exact symbol lookups. If a user asks about a specific class, we don't guess. We fetch the exact definition from the symbol table.

The Pipeline

Our architecture is a series of Go microservices. We chose Go because we need the concurrency for high-throughput cloning and parsing.

1. Ingestor: Clones the repo and triggers a webhook. 2. Grapher: Runs Tree-sitter and builds the symbol map. 3.

Embedder: Generates vectors using text-embedding-3-small. It's cheap and honestly better than the large version for code. 4. Ranker: The cross-encoder that weeds out the noise.

This runs in about 45 seconds for a medium-sized repo (50k LOC). Larger repos take longer, but we parallelize by directory to keep it under 5 minutes.

Stop trying to be clever

The biggest mistake we made early on was trying to let the LLM decide which files to read. Agents are slow and they loop forever.

Deterministic retrieval is better. If you can use regex or an AST to find a file, do it. Don't waste $0.05 on an LLM call to find a file path that find . -name could have found in 2ms.

Treat the LLM as a sophisticated reasoning engine, not a file system. Feed it the right data, and it works. Feed it junk, and you get a fancy autocomplete that breaks your production environment.

Results from the field

We tested this on the denoland/deno repo. A standard RAG setup answered 22% of architectural questions correctly. With AST-based chunking and symbol mapping, that jumped to 74%.

The remaining gap isn't a retrieval problem. It's a logic problem. We're working on that next.

Don't trust any tool that claims to "understand" your code if they can't explain their parsing strategy. If they aren't using an AST, they're just guessing.