HubSpot now runs 20 billion+ vectors across 200+ indexes and 140+ clusters, serving 38+ inte...
HubSpot published details this week about the internal platform they use to serve semantic search across the company, and the numbers are worth pausing on: more than 20 billion vectors spread across 200 plus indexes and 140 plus clusters, all consumed by 38 plus internal teams through a single Vector as a Service offering. The story matters less because of the raw scale and more because of what it reveals about where production RAG systems actually break.
The team started on Qdrant deployed through Helm charts with humans running the operational work. That model fell apart quickly. The answer was not a different vector database or a fancier embedding model. It was pulling cluster lifecycle into Kubernetes operators that handle provisioning, shard rebalancing, replication recovery, and scaling automatically. Spin up time for a new index dropped from hours to minutes. That single change is what made the platform usable across dozens of product teams instead of one or two.
The technical implication is worth sitting with. Most public conversation about retrieval augmented generation still centers on which embedding model to use, which vector database benchmarks best, or how to tune chunk sizes. HubSpot's experience says the bleeding is happening one layer down, in the retrieval control plane. If every team building a RAG feature has to file a ticket to create an index, hand tune shards for their workload, and wake someone up when a replica fails at two in the morning, then your semantic search team becomes the bottleneck long before embedding quality does. The failure mode is organizational, not algorithmic.
The pattern HubSpot landed on is familiar from other platform engineering domains: treat vector infrastructure as a product, wrap it in operators, expose it through a self serve interface, and let application teams consume it without needing to understand the operational internals. That is what makes 38 teams shipping retrieval features tractable without 38 separate on call rotations. It is the same shape as internal Kubernetes platforms, internal databases as a service, and internal feature stores. Vector infrastructure is just the newest member of that family.
What is worth watching next is whether managed vector database vendors move to close this gap themselves, or whether large engineering organizations keep building this platform layer in house. The center of gravity in RAG has shifted from picking a vector database to operating it as multi tenant infrastructure, and the companies that recognize this early will ship retrieval features at a pace that others cannot match, regardless of which model or database sits underneath.
Originally posted on LinkedIn.