Parsing at 50,000 lines per second

Most code analysis tools are slow because they treat your repository like a homework assignment. They try to compile the world before telling you where the bug is. That’s a mistake.

When we started git11, we realized that waiting three minutes for a CI runner to boot up is a death sentence for developer flow. If the feedback isn't instant, you’ve already checked Twitter.

Real-time analysis isn't about being smart. It's about being fast enough to be wrong, then fixing it before the user notices.

The lie of the Language Server Protocol

LSPs are great for IDEs, but they are a bottleneck for fleet-wide analysis. They are heavy, memory-hungry, and stateful.

Starting a TypeScript language server for a 500k LOC repo takes ten seconds. That's ten seconds of dead air. We can't afford that.

We ditched the idea of running full compilers. Instead, we use Tree-sitter. It’s a fast, incremental parsing library that generates concrete syntax trees (CST) without needing a build environment.

Tree-sitter doesn't care if your node_modules is missing or if your C++ headers are broken. It parses what it sees. In our world, a partial truth delivered in 20ms is better than a perfect truth delivered in 20 minutes.

Content-addressable graphs

We don't store files. We store hashes. If you rename a file, we shouldn't have to re-parse it.

Every function, class, and method is hashed into a Merkle tree. When you push a commit, we diff the hashes. If auth_service.py changed but validate_token() stayed the same, we reuse the existing analysis metadata.

This is why git11 feels instant. We aren't analyzing your whole repo; we’re analyzing the 40 lines of delta and stitching them into a pre-computed graph.

We use Rust for this. Specifically, we use dashmap for high-concurrency lookups and smallvec to keep things off the heap. Garbage collection is a luxury we don't have when we're trying to hit sub-100ms latency.

The false promise of Regex

Grepping for patterns is for amateurs. It misses context. A string that looks like an API key in a comment is different from one in a config file.

We built a query layer on top of our CSTs. We use a DSL that looks a lot like S-expressions. It allows us to ask things like: "Find all calls to db.execute where the first argument is not a literal string."

This catches SQL injection. It also runs across 1,000 files in under a second because we parallelize the search across every core your CPU can give us.

If you're using Python or Node for this logic, you've already lost. We use Rayon to shard the workload. The OS scheduler is better at managing threads than you are; let it do its job.

Memory is the enemy

Modern machines have 64GB of RAM, and yet most tools still OOM on large repos. Why? Because they hold the entire AST in memory.

We stream. We parse a file, extract the symbols, write the edges to a persistent key-value store (RocksDB), and drop the tree.

RocksDB is tuned for heavy writes. We use it to store our symbol index. This allows us to handle repos that are larger than the available RAM on the analysis node.

If you're building a tool and your memory usage grows linearly with the size of the codebase, you've built a toy, not a tool.

Context injection for AI

When you ask an AI about your code, it needs context. But dumping 10,000 files into a prompt is a disaster. It's expensive and the model gets confused.

We use our symbol graph to perform "Ranked Retrieval." If you're asking about a bug in payments.go, we don't just send that file.

We find every call site of the functions in that file. We find the interface definitions. We find the SQL schema linked to the structs.

We pack this into a dense context window. It's the difference between asking a librarian for "a book about ships" and asking for "the blueprint for the Titanic's engine room."

Stop building for the happy path

Most analysis tools fail because they assume the code is valid. It rarely is.

Developers write broken code all day. Your parser needs to be error-tolerant. Tree-sitter’s ability to recover from syntax errors is its most important feature.

If a dev is halfway through writing a function, we should still be able to map the variables they've already declared. If your tool crashes on a missing semicolon, throw it away.

The takeaway

Speed is a feature, but it's also a constraint that dictates your entire architecture.

Rust for the heavy lifting. Tree-sitter for the parsing. RocksDB for the state.

No LSPs. No heavy runtimes.

If it takes more than a second to see the impact of a code change, the tool is getting in the way. Build for the developer who is tired, caffeinated, and in a hurry. They are the ones who actually ship things.