Agents need vector search more than RAG ever did
What's the role of vector databases in the agentic AI world? That's a question that organizations have been coming to terms with in recent months. The narrative had real momentum. As large language models scaled to million-token context windows, a credible argument circulated among enterprise architects: purpose-built vector search was a stopgap, not infrastructure. Agentic memory would absorb the retrieval problem. Vector databases were a RAG-era artifact. The production evidence is running the other way. Qdrant, the Berlin-based open source vector search company, announced a $50 million Series B on Thursday, two years after a $28 million Series A. The timing is not incidental. The company is also shipping version 1.17 of its platform. Together, they reflect a specific argument: The retrieval problem did not shrink when agents arrived. It scaled up and got harder. "Humans make a few queries every few minutes," Andre Zayarni, Qdrant's CEO and co-founder, told VentureBeat. "Agents make hundreds or even thousands of queries per second, just gathering information to be able to make decisions." That shift changes the infrastructure requirements in ways that RAG-era deployments were never designed to handle. Why agents need a retrieval layer that memory can't replace Agents operate on information they were never trained on: proprietary enterprise data, current information, millions of documents that change continuously. Context windows manage session state. They don't provide high-recall search across that data, maintain retrieval quality as it changes, or sustain the query volumes autonomous decision-making generates. "The majority of AI memory frameworks out there are using some kind of vector storage," Zayarni said. The implication is direct: even the tools positioned as memory alternatives rely on retrieval infrastructure underneath. Three failure modes surface when that retrieval layer isn't purpose-built for the load. At document scale, a missed result is not a latency problem — it is a quality-of-decision problem that compounds across every retrieval pass in a single agent turn. Under write load, relevance degrades because newly ingested data sits in unoptimized segments before indexing catches up, making searches over the freshest data slower and less accurate precisely when current information matters most. Across distributed infrastructure, a single slow replica pushes latency across every parallel tool call in an agent turn — a delay a human user absorbs as inconvenience but an autonomous agent cannot. Qdrant's 1.17 release addresses each directly. A relevance feedback query improves recall by adjusting similarity scoring on the next retrieval pass using lightweight model-generated signals, without retraining the embedding model. A delayed fan-out feature queries a second replica when the first exceeds a configurable latency threshold. A new cluster-wide telemetry API replaces node-by-node troubleshooting with a single view across the entire cluster. Why Qdrant doesn't want to be called a vector database anymore Nearly every major database now supports vectors as a data type — from hyperscalers to traditional relational systems. That shift has changed the competitive question. The data type is now table stakes. What remains specialized is retrieval quality at production scale. That distinction is why Zayarni no longer wants Qdrant called a vector database. "We're building an information retrieval layer for the AI age," he said. "Databases are for storing user data. If the quality of search results matters, you need a search engine." His advice for teams starting out: use whatever vector support is already in your stack. The teams that migrate to purpose-built retrieval do so when scale forces the issue. "We see companies come to us every day saying they started with Postgres and thought it was good enough — and it's not." Qdrant's architecture, written in Rust, gives it memory efficiency and low-level performance control that higher-level languages don't match at the same cost. The open source foundation compounds that advantage — community feedback and developer adoption are what allow a company at Qdrant's scale to compete with vendors that have far larger engineering resources. "Without it, we wouldn't be where we are right now at all," Zayarni said. How two production teams found the limits of general-purpose databases The companies building production AI systems on Qdrant are making the same argument from different directions: agents need a retrieval layer, and conversational or contextual memory is not a substitute for it. GlassDollar helps enterprises including Siemens and Mahle evaluate startups. Search is the core product: a user describes a need in natural language and gets back a ranked shortlist from a corpus of millions of companies. The architecture runs query expansion on every request - a single prompt fans out into multiple parallel queries, each retrieving candidates from a different angle, before results are combined and re-ranked. That is an agentic retrieval pattern, not a RAG pattern, and it requires purpose-built search infrastructure to sustain it at volume. The company migrated from Elasticsearch as it scaled toward 10 million indexed documents. After moving to Qdrant it cut infrastructure costs by roughly 40%, dropped a keyword-based compensation layer it had maintained to offset Elasticsearch's relevance gaps, and saw a 3x increase in user engagement. "We measure success by recall," Kamen Kanev, GlassDollar's head of product, told VentureBeat. "If the best companies aren't in the results, nothing else matters. The user loses trust." Agentic memory and extended context windows aren't enough to absorb the workload that GlassDollar needs, either. "That's an infrastructure problem, not a conversation state management task," Kanev said. "It's not something you solve by extending a context window." Another Qdrant user is &AI, which is building infrastructure for patent litigation. Its AI agent, Andy, runs semantic search across hundreds of millions of documents spanning decades and multiple jurisdictions. Patent attorneys will not act on AI-generated legal text, which means every result the agent surfaces has to be grounded in a real document. "Our whole architecture is designed to minimize hallucination risk by making retrieval the core primitive, not generation," Herbie Turner, &AI's founder and CTO, told VentureBeat. For &AI, the agent layer and the retrieval layer are distinct by design. "Andy, our patent agent, is built on top of Qdrant," Turner said. "The agent is the interface. The vector database is the ground truth." Three signals it's time to move off your current setup The practical starting point: use whatever vector capability is already in your stack. The evaluation question isn't whether to add vector search — it's when your current setup stops being adequate. Three signals mark that point: retrieval quality is directly tied to business outcomes; query patterns involve expansion, multi-stage re-ranking, or parallel tool calls; or data volume crosses into the tens of millions of documents. At that point the evaluation shifts to operational questions: how much visibility does your current setup give you into what's happening across a distributed cluster, and how much performance headroom does it have when agent query volumes increase. "There's a lot of noise right now about what replaces the retrieval layer," Kanev said. "But for anyone building a product where retrieval quality is the product, where missing a result has real business consequences, you need dedicated search infrastructure."
No discussion yet. Be the first to share your thoughts!