Ask a coding agent "where do we handle authentication?" and watch what a plain grep does with it. It searches for the literal string authentication. It misses the file where the logic actually lives, because someone named it IdentityMiddleware, wired it through a TokenValidator, and never once wrote the word you searched for. Meanwhile it happily returns forty matches in comments, test fixtures, and a changelog. The tool did exactly what you asked. It just doesn't know what you meant.
That gap — between the tokens you type and the meaning you intend — is the whole reason semantic search has become table stakes for AI coding assistants. Cursor recently published its results from building one, and the interesting part isn't that embeddings help. It's how much, and the training trick that made their model earn its keep.
Lexical search hits a wall
Regex-based tools are fast, exact, and completely literal. That's a strength when you know the symbol you're hunting for and a liability when you're reasoning about behavior. An agent working a real task doesn't start with the symbol name — it starts with an intent ("find the retry logic," "where's the rate limiter") and has to discover the vocabulary the codebase actually uses. Grep can't bridge synonyms, can't follow a concept across files that spell it differently, and drowns useful hits in noise on any common term.
Semantic search inverts the model. Instead of matching characters, it matches meaning. Both the query and every chunk of code get mapped into the same high-dimensional vector space by an embedding model, and retrieval becomes a nearest-neighbor lookup: which code vectors sit closest to the query vector. "Where do we handle authentication?" lands near the token-validation code regardless of what it's named, because the model learned those two things mean similar things.
The part that actually matters: training on agent traces
Here's where Cursor's writeup gets specific, and where a generic off-the-shelf embedding model would leave value on the table. A model trained on "generic code similarity" learns that two functions look alike. But that's not the question an agent is asking. The agent's question is: given where I am in this task, which code would help me next?
So they trained the embedding model on that signal directly. Their pipeline uses agent session traces — real runs where the agent searched, opened files, and worked through a task — as training data. An LLM then ranks which content would have actually been helpful at each step, and the embedding model is trained to align its similarity scores with those rankings. As Cursor puts it, the model learns from "how agents actually work through coding tasks, rather than relying on generic code similarity."
That's a meaningful reframing. The retrieval target isn't "similar code," it's "code that unblocks the next action." Same architecture, completely different objective function.
What it bought them
Cursor measured this two ways, and both are worth stealing as a template.
Offline, they built a proprietary benchmark — "Cursor Context Bench," a dataset with known correct answers — and evaluated frontier models with and without semantic search:
- 12.5% higher accuracy answering questions on average
- A range of 6.5% to 23.5% depending on the model
- Improvement across every frontier model tested, not just their own
Online, in a live A/B test, the effects are smaller but tell you something offline numbers can't:
- +0.3% code retention overall with semantic search available
- +2.6% for large codebases — those with 1,000+ files
- A 2.2% increase in dissatisfied follow-up requests when semantic search was turned off
The large-codebase number is the tell. Semantic search barely moves the needle on a toy repo where grep can brute-force everything anyway. Its value scales with the size of the haystack — exactly where AI assistants otherwise fall apart.
The plumbing is a database problem
Retrieval quality is the model; retrieval at all is an indexing pipeline. This is where the vector-database machinery lives, and it's the less glamorous half of the system.
The flow looks roughly like this:
local repo ──chunk──▶ embed ──▶ [vector + line range + obfuscated path]
│
▼
remote vector DB
query ──embed──▶ nearest-neighbor search ──▶ paths + line ranges
│
▼
client reads actual code from local disk
Code is chunked locally, each chunk is embedded, and the vectors — alongside metadata like line numbers and obfuscated file paths — are written to a remote vector store (Cursor uses Turbopuffer). At query time the client embeds the query, runs the similarity search, gets back paths and line ranges, and reads the real source from the local filesystem. The server holds vectors and offsets, not your source.
Two details make this practical at scale. Cursor builds a Merkle tree of file hashes so that re-syncing an index means diffing hashes and uploading only what changed, not re-embedding the repo every five minutes. And path obfuscation (splitting paths at / and .) plus a policy of not persisting raw code past the life of a request is what makes "index my private monorepo in the cloud" something a security team will actually sign off on.
If you're building this yourself
The lesson isn't "add embeddings." It's three concrete decisions:
- Don't replace grep — pair it. Cursor is explicit that their agent leans on both, and "the combination of these two leads to the best outcomes." Lexical search wins on exact symbols; semantic wins on intent. Route to both and let the agent reconcile.
- Train on the task, not on similarity. If you can capture traces of what your agent opened and what actually helped, that ranking signal is worth more than any generic code-similarity objective.
- Budget for the index, not just the model. Incremental sync, a real vector store, and a privacy story are the difference between a demo and something that survives a 50,000-file repo.
Grep answers "where is this string." Semantic search answers "where is this idea" — and on a large codebase, that's the question your agent was actually asking the whole time.
Sources: Semantic Search for Coding Agents — Cursor, Securely indexing large codebases — Cursor