Search is dead, long live agentic search
tl;dr
Agents search differently than humans do: in parallel, with far more expressive queries, and patiently. But they are also naive — they (mostly) believe everything they read. That turns retrieval into a context hygiene problem, and it breaks the relevance metrics of traditional search systems.
How agents search
I attended Berlin Buzzwords this year, and if there was one buzzword, it was "agentic search". Since I've worked on search myself, here is a short write-up of what stuck with me.
The clearest way to put it, is a table:
| Human | Agent |
|---|---|
| searches sequentially, 2-3 keywords | searches in parallel, with filters, facets, synonyms |
| ignores the docs | happily reads the docs — if you give them to it |
| scans the result list top to bottom, stops around 10 | reads the whole list at once, position 47 is as visible as position 1 |
| spots contradictions between hits intuitively | takes every hit at face value |
| knows which sources to trust from experience | has no sense of source authority unless you hand it over as metadata |
| expects results in < 1s | User expects a final answer in < 1 minute |
That last row is interesting! We get a whole bunch of latency budget for free. I certainly don't enjoy a searchbar that takes several seconds to present me with results, but an agent doesn't really care, as long as it ends up with solid hits.
One drop of ink discolors your whole glass
Problem: A paper from May 2026, The First Drop of Ink, varied the share of hard distractors in a fixed-length context — documents that look relevant but are misleading. The degradation is non-linear: accuracy collapses within the first few percent, and barely drops further after that. In their setup, adding 10% hard distractors pulls roughly three quarters of the attention away from the documents that actually matter. One drop of ink colors the entire glass; the second one hardly matters.
Removing some hard distractors doesn't really help, you have to remove them all. And the distracting hits are the ones your retrieval ranks highly. In legal (re)search, where I used to think about this problem, they're everywhere, and the document itself doesn't tell you on its own:
- Overturned case law. The obsolete headnote reads just as authoritative as the current one.
- Minority opinions that rarely win in court is not false, just not the prevailing view.
- Reversed decisions. The reversal lives in a different document.
These distractors can only be recognized by their contextual data, i.e. metadata. If a decision was reversed or overruled, this needs to be in the metadata of that document. So we need to develop ideas to also update metadata of older documents with a new run of our processing pipeline.
What it costs to keep the context clean
The systems presented by various providers/startups at Berlin Buzzwords filter hits through a multi-stage pipeline before the model ever sees them: embed each hit on the fly, rerank it against the query, have a small model summarize it, then hand out results piecewise on request, reformulate, repeat.
But, steps 2 to 4 run per hit, not per search. Your bill scales with hits × rounds, not with users. And the summarization is a lossy compression step — it can cut exactly the nuance that mattered and produce a hit that looks clean and still is misleading.
What search engines need to change (if they want to cater for agents)
- Rich interfaces - filters, facets, anything most users won't touch
- Metadata on hits and result lists - everything that lets a model judge relevance without reading the hit.
- Good API docs - agents loooove documentation
The part I don't have an answer for
If you get all that right, search becomes a hill climbing problem: build, measure, tweak, measure, improve.
Except hill climbing needs an objective function, and ours is built for humans. Take NDCG, NDCG@10 smuggles in three assumptions: position 1 counts more than position 10, nothing past 10 matters, and each hit is judged on its own. None of this is true for agents doing search. A hard distractor is problematic because it sits next to the correct hits and steals attention from "good" hits. A per-document relevance label can't express this nuance. It knows "relevant" and "irrelevant", but not "correct, but superseded".
What works today are LLM-as-a-judge approaches and hand-curated benchmarks. But those are the opposite of what hill climbing needs. Both are expensive to prepare (human time) or expensive to run (inference time). On top of that, the ground truth isn't static: case law changes, and today's golden set entry becomes tomorrow's hard distractor.
Summary
Search isn't dead, but its users are changing. Most of what we build and tune assumes a human on the other end of the API — reading top to bottom, giving up after ten results, quietly discarding what doesn't add up. An agent does none of that.
The open question I'm stuck on: how do we detect hard distractors relative to a concrete query, and filter them?
Resources: