Inside git11's AI architecture

{ "title": "Naive RAG is a lie: How git11 actually maps 10M+ lines of code", "subtitle": "Slapping a vector DB on a repository doesn't work. Here is how we built a context engine that doesn't hallucinate your imports.", "content": "Most AI tools treat your codebase like a collection of text documents. They aren't. \n\nA codebase is a graph of dependencies, a history of bad decisions, and a high-stakes puzzle of types. If you just chunk auth_service.go into 512-token segments and shove them into Pinecone, your LLM will fail. It won't know that the User struct used on line 40 is defined in a shared internal library three subdirectories away.\n\nWe spent six months breaking things before we realized: context window size is a vanity metric. What matters is which bytes you choose to put in it.\n\n## The failure of basic embedding\n\nStandard RAG (Retrieval-Augmented Generation) is built for FAQs, not logic. When you ask a bot, \"Why is this database connection leaking?\", a vector search looks for the word \"leak.\" \n\nIt misses the defer statement you forgot five functions deep in a different file. It misses the weird middleware config in .git11/config.yaml. \n\nWe stopped using generic semantic search as our primary retrieval method. Instead, we built a three-layer pipeline: structural parsing, symbol mapping, and then (and only then) vector-weighted re-ranking.\n\n## Step 1: Treesitter is the source of truth\n\nEvery time you push a commit, we don't just index text. We run tree-sitter over every file. \n\nThis gives us a concrete syntax tree (CST). We know exactly what is a function definition, what is a call site, and what is a comment. We extract these into a custom intermediate representation we call a CodeGraph.\n\n``rust\n// Internal representation snippet\nstruct Node {\n id: SymbolID,\n kind: SymbolKind, // Struct, Method, Interface\n location: Span,\n refs: Vec<SymbolID>,\n}\n`\n\nIf you change a signature in api/v1/users.ts, we know every single location in the repo that just broke. We don't guess. We trace the graph.\n\n## Step 2: The Global Symbol Table\n\nRunning ls -R isn't a strategy. To provide context to an LLM, we maintain a global symbol table for the entire repository. \n\nWhen the model looks at a block of code, we calculate the \"Local Density\" of symbols. If it sees PaymentProcessor, we don't just grab the file it's in. We grab the interface definition, the two most common implementations, and the last three PRs that touched it.\n\nWe found that providing the interface is 10x more useful than providing the implementation. It keeps the prompt clean and prevents the model from getting lost in the weeds of a 2,000-line .java` file.\n\n## Step 3: Weighted context stitching\n\nWe have a 128k context window to fill. We don't fill it with the \"most similar\" chunks. We use a priority queue based on three factors:\n\n1. Distance in the Graph: How many imports away is this code?\n2.

Temporal Relevance: Was this file changed in the last 48 hours?\n3. Type Overlap: Does this chunk use the same types as the user's query?\n\nIf you’re working on a bug in billing.py, we prioritize the stripe_client.py and the recent migration file over a generic utils.py that happens to have the word \"billing\" in it.\n\n## Why LLMs hate your folder structure\n\nHumans like folders. Computers don't care. LLMs specifically struggle with deep nesting. \n\nWe flatten the context. When we send code to the model, we rewrite the file paths and strip the boilerplate. We inject \"breadcrumb\" comments that tell the model exactly where it is in the project hierarchy without wasting tokens on repetitive directory names.\n\nWe also use a custom tokenizer filter. We found that standard OpenAI tokenizers are incredibly inefficient at handling Python indentation. We pre-process the whitespace to save about 12% of total token count without losing structure.\n\n## The Late Interaction trick\n\nCross-encoders are expensive but necessary. If you use a bi-encoder for everything, you get fast, garbage results. \n\nWe use a fast vector search (Qdrant) to get the top 100 candidates. Then, we run a ColBERT-style late interaction model to re-rank those 100. This smaller model looks at the fine-grained relationship between the query tokens and the code tokens.\n\nIt’s the difference between finding a file that looks right and finding the one line of code that is right.\n\n## Don't trust the model to read\n\nModel hallucination usually happens when the model is forced to ignore irrelevant data. If you give it 50 files, it will try to find a way to link them all, even if 45 are noise.\n\nOur \"Janitor\" agent runs before the main prompt. Its only job is to look at the retrieved context and say \"This is trash, delete it.\" It prunes the context by another 30% before the user ever sees a response. \n\nAggressive pruning beats a large window every single time.\n\n## Engineering for latency\n\nRunning all of this—Tree-sitter, symbol mapping, vector search, and re-ranking—takes time. Users won't wait 10 seconds for a chat response. \n\nWe offload the heavy lifting to our Rust-based indexing engine. The index isn't a static file; it's a live actor in our system. When you type in the editor, we are already pre-calculating the symbol density for your current cursor position.\n\nBy the time you hit \"Enter,\" the context is already staged in Redis.\n\n## The takeaway\n\nStop optimizing your prompts. Optimize your data pipeline. \n\nIf your AI doesn't understand your code's graph, it’s just a glorified autocorrect. Build the graph first, index the symbols second, and only use embeddings to bridge the gap in natural language. \n\nBetter context always beats a better model. Spend your time on the parser, not the prompt.", "tags": ["architecture", "rust", "llms", "indexing"], "category": "technical", "read_time": 7 }