Elastic's '7x Faster Than Qdrant' Benchmark: What It Actually Shows

Elastic’s “7x Faster Than Qdrant” Benchmark: What It Actually Shows

Elastic published a benchmark  this month claiming its new DiskBBQ vector index is up to 7x faster than Qdrant. It’s a thorough piece, with full parameter sweeps, a published dataset, an open-source benchmarking tool, and a methodology section that reads like it should be taken seriously. That’s more rigor than most vendor benchmarks bother with, and for that it deserves some credit.

It is also a benchmark that stacks a new Elastic feature against a competitor in a storage and memory configuration that favors the new feature. That does not make the result meaningless. It means the result should be read as a specific engineering claim about one topology rather than a universal verdict on the two systems.

The claim, and what’s actually behind it

The headline: Elasticsearch DiskBBQ delivers up to 7x higher throughput than Qdrant at comparable recall on network-attached storage.

The “7x” comes from one row of a larger table. The full matched-recall comparison looks like this:

So “up to 7x” is real, but it’s the ceiling, not the average, and it only shows up once you’re above ~0.93 recall. At the lower end of the tested range it’s 2.3x. A headline built on the best cell in the table isn’t lying, but it’s not a neutral description of the result either. This is the same move any vendor makes when picking which number goes above the fold.

The storage topology was chosen to produce this result

This is the part worth sitting with. Elastic explicitly used network-attached persistent storage, a GCP Hyperdisk Balanced volume at baseline performance, with no additional IOPS or throughput provisioning, and says so directly: “Qdrant often performs better on local NVMe.” They also sized the cluster nodes (7 vCPU, 26GB RAM per pod) specifically so the working set would not comfortably fit in memory, which is exactly the condition under which a disk-optimized index shows off.

Qdrant’s own documentation is blunt about this same tradeoff: local SSD delivers roughly 183,000 read IOPS versus roughly 6,300 IOPS on network-mounted storage in their published comparisons, a 25-30x gap. That difference matters because Qdrant’s rescoring step does random reads against full-precision vectors on disk. When those reads hit local NVMe instead of network-attached storage, the cost of each rescore drops sharply. Elastic did not discover a hidden weakness. They tested a storage-bound configuration that is very much in the zone where DiskBBQ is meant to shine.

Calling this “the topology most managed-cloud deployments actually use” is doing a lot of work. It’s true that many Kubernetes/managed-cloud setups default to network block storage. It’s also true that anyone running a disk-resident ANN index at scale and reading Qdrant’s docs would often provision local NVMe or size RAM to hold the quantized index, because Qdrant tells you to. The benchmark is still useful, but it is best read as a stress test for one storage topology and one licensing tier, not as a universal verdict on what either system looks like in every deployment.

It’s an Enterprise feature vs. a default config

bbq_disk is an Elastic Enterprise-licensed feature. Elastic’s own self-managed pricing page  lists DiskBBQ under Enterprise, not the Free tier or Platinum. It is gated behind the top license level. The benchmark post says as much, almost in passing. They used it “because the goal… was to compare the strongest disk-oriented vector search configuration available in each engine… rather than licensing tiers.” Fair enough as an engineering choice, but a reader skimming “Elasticsearch beats Qdrant 7x” has no reason to know the winning configuration requires a paid license tier, and the post never turns that into a cost comparison, only a throughput-per-node one.

That license isn’t free, and it isn’t small. Elastic’s actual GSA schedule pricing (public federal contract pricing, not a rumor) lists self-managed Enterprise at $9,788.41 per year per 64GB “Enterprise Resource Unit” under the current OneGov promotional rate. The hardware figures below use Google Cloud’s published on-demand N4 machine-type pricing from the general-purpose VM pricing page for the compute layer of the benchmark cluster. They do not include storage, discounts, or region-specific price differences. The Elastic license figures come from Elastic’s published pricing page.

Here is the comparison in three parts:

1. What Elastic actually tested
2. What that costs
3. What a properly sized Qdrant deployment costs

1) What Elastic actually tested

Elastic benchmarked on GCP using three n4-standard-8 nodes, with each pod allocated 7 vCPUs and 26 GB of RAM, and 200 GiB Hyperdisk Balanced volumes at baseline performance. In other words, the test was a disk-backed, network-attached-storage setup that Elastic chose because it is a common managed-cloud topology and because it puts storage efficiency directly on the query path.

Elastic did not run a separate local-NVMe Qdrant benchmark in this post; the published comparison was explicitly on network-attached persistent storage, with the caveat that Qdrant tends to do better on local NVMe. The row above is a more realistic Qdrant sizing based on Qdrant’s own guidance, not a second benchmark from Elastic.

2) What that costs

The benchmark’s own cluster consists of three nodes at 26GB RAM each, for 78GB total, and needs at least two of those units, and possibly three depending on whether Elastic licenses by aggregate cluster RAM or per node:

Source: Google Cloud general-purpose VM pricing for N4 machine types (on-demand, us-central1) plus Elastic self-managed Enterprise pricing. The compute number excludes storage and discounts.

3) What a properly sized Qdrant deployment costs

By contrast, a more realistic Qdrant deployment, one that follows Qdrant’s own guidance around local NVMe and enough memory to avoid the disk-rescoring path, would require a larger memory footprint than the published 26GB-per-pod configuration. A rough, illustrative way to think about it is to move from three n4-standard-8 nodes to three n4-standard-16 nodes, which would roughly double the node cost while keeping the same basic topology.

Source: Google Cloud general-purpose VM pricing for N4 machine types (on-demand, us-central1). This is the compute layer only; it excludes storage and discounts.

That is still a smaller incremental cost than paying for an Enterprise license tier on top of the cluster, and it does not recur as a percentage tax on the whole deployment. Two systems can have very different economics even when one wins on raw QPS, and here the economics are materially less favorable to the Elastic configuration once you include the licensing layer.

Final cost comparison

If you collapse the comparison to a single bottom line, the economics look like this:

That makes the comparison fairly stark: the more realistic Qdrant configuration is still cheaper than the Elastic setup once you include the Enterprise licensing layer, even though the Elastic benchmark is the one that wins on the headline throughput table.

One dataset, one dimensionality, one hardware shape

The entire comparison rests on a single corpus: 21 million 768-dimensional e5 embeddings of Wikipedia passages, on one node size, with recall tuned by sweeping exactly one parameter per engine (visit_percentage for Elasticsearch, hnsw_ef for Qdrant, oversampling pinned at 1 throughout). That’s a legitimate way to isolate a variable, but it’s also a narrow slice of the design space. Different embedding dimensionality, different corpus size relative to RAM, different oversampling/rescoring settings, or a different quantization depth could all move the needle, and none of those were tested. The post also notes 2-bit quantization was used because Qdrant “couldn’t reach the target recall range with 1-bit binary quantization,” which is a tell that the comparison bent around whichever knobs made both engines land in a similar recall band, not around a fixed, independently chosen configuration for each.

None of this makes the underlying methodology dishonest. Matched-recall comparison instead of matched-parameter comparison is actually the right instinct, since visit_percentage and hnsw_ef aren’t equivalent controls. Credit where due. But “correct methodology” and “representative result” are different claims, and the post asks you to accept the second on the strength of the first, even though the benchmark is still informative if you read it as a bounded point estimate rather than a general-purpose verdict.

What’s actually true, and worth taking from this

Strip away the leaderboard framing and there’s a real engineering point underneath: on network-attached storage, an index format that keeps the approximate search phase entirely in a compact quantized representation and defers full-precision reads to a minimum will beat an index that has to do random-access rescoring against full vectors on the same disk. That is a useful result, but it is also a bounded one. It tells you something important about one storage topology, one licensing tier, and one set of hardware assumptions. It does not settle the broader question of which system is better in every deployment.