Perplexity has rebuilt a critical storage layer underneath its AI search stack, replacing a managed DynamoDB-based hot store with a purpose-built distributed system called CobbleDB. In production measurements reported by the company, median batch-read latency fell from 31.4 milliseconds to 5.60 milliseconds, while p99 latency dropped from 123 milliseconds to 24.2 milliseconds.
The architecture is described in Perplexity’s official September 14 technical report, “CobbleDB: Rebuilding AI Search Storage for Lower Latency and Cost.” Perplexity says the new system serves prepared representations of web pages—pre-chunked passages together with vector embeddings—that can be retrieved after search and ranking and passed onward to the model.
That makes CobbleDB part of the infrastructure beneath sourced AI answers. It does not itself determine whether a citation is correct or decide which source deserves to appear, but it accelerates the stage where Perplexity retrieves the prepared source content needed by downstream answer generation.
AI search stores passages for models, not pages for humans
Perplexity’s redesign begins with a distinction between traditional web search and AI-native search. A conventional search engine can rank documents and send the user to a page. An answer engine needs to retrieve content that a language model can consume directly.
Perplexity says its processing pipeline cleans raw HTML, divides each page into semantically coherent passages, computes embeddings and stores the passages and embeddings together. At query time, the serving pipeline identifies relevant pages and retrieves their prepared representations so the model can work with the most useful text.
This changes the storage workload. Instead of simply looking up a URL or returning a document identifier, the system repeatedly reads relatively large records containing model-ready chunks and vector representations.
The database therefore sits on a latency-sensitive path between search retrieval and model generation. Every extra millisecond spent retrieving those prepared records can add to the time before the answer engine has the evidence it needs.
The old architecture used DynamoDB as the hot store
Perplexity previously kept prepared pages in a dedicated DynamoDB table for query-time reads, with full page content and metadata stored separately. The managed service gave the company high availability without requiring it to operate a distributed database itself.
As the search workload grew, however, Perplexity says three limitations became more important: usage-based costs increased with repeated reads and writes, the team lacked low-level control over placement and caching, and document processing was coupled too closely to writes into the live serving store.
A single Search API request can contain roughly 100 to 120 page keys that need to be fetched. Perplexity says the data-retrieval process divides that work into smaller batches of 10 to 20 page keys, with an average item size of about 50 KB.
At AI-search scale, repeatedly reading records of that size turns storage behavior into a major infrastructure concern rather than a background database choice.
CobbleDB is optimized for one narrow operation
Perplexity did not build CobbleDB as a general-purpose database. The system is optimized for a specific hot-path operation: given a batch of page identifiers, return the prepared page content quickly.
Keys are hashed page identifiers derived from URLs. Values contain the processed representation of the page, including pre-chunked passages and vector embeddings for those chunks.
Data is partitioned across nodes, with three replicas of each partition placed on different machines. Each data node uses RocksDB as its embedded key-value engine, while cached data can be served from memory and uncached reads use local NVMe storage.
A stateless query router hashes keys to partitions and sends requests to nodes in parallel. Perplexity says the router prefers a replica in the same availability zone when possible, reducing cross-zone latency.
Batch retrieval is where the largest latency gains appear
After migrating the hot store, Perplexity reports that median production batch-read latency fell from 31.4 ms to 5.60 ms. The p90 dropped from 56.7 ms to 9.77 ms, while the p99 fell from 123 ms to 24.2 ms.
Those figures correspond to improvements ranging from 5.08× to 5.80× across the reported percentiles. The improvement therefore appears not only in typical requests but also in the slower tail that can disproportionately affect perceived search latency.
Perplexity says the production measurements come from batch reads requesting roughly 10 to 15 keys, with an average payload item size of 50 KB. Both systems were queried at approximately 200,000 requests per second.
The company also reports load tests reaching 500,000 requests per second without performance degradation.
The production comparison is before-and-after, not a simultaneous controlled test
Perplexity explicitly qualifies the headline latency figures. The production measurement is observational: DynamoDB and CobbleDB served real traffic at different times rather than processing the exact same live requests simultaneously under controlled conditions.
The company says other aspects of the configuration were kept constant to isolate CobbleDB’s contribution, but the temporal difference means the production numbers should not be interpreted as a perfect A/B experiment.
To address that limitation, Perplexity also ran a synthetic performance benchmark. It queried both systems using batches of 10 to 15 keys and values ranging from 100 bytes to 100 KiB, approximating the request pattern observed in production.
This methodological caveat matters because infrastructure claims can look more definitive than the underlying measurement supports. Perplexity is reporting a substantial observed improvement, while also acknowledging that its strongest real-world comparison is a migration before-and-after rather than simultaneous controlled traffic.
RocksDB MultiGet is built for the batch pattern Perplexity needs
Once search and ranking produce a candidate set of relevant pages, CobbleDB receives a batch request for their prepared content. Each data node can use RocksDB’s MultiGet operation to retrieve multiple keys together.
This matches Perplexity’s workload more directly than a generic database interface designed to support many different access patterns. The company can optimize caches, local storage and replica routing around repeated batch reads.
If one node responds slowly, the system can issue a request to another replica of the partition. That hedging mechanism helps reduce tail latency, which is particularly important when a batch cannot complete until its slowest required data arrives.
Perplexity can also omit database features its hot store does not require. It says CobbleDB does not need transactions or perfectly synchronized replicas because a short delay between a write and read availability is acceptable for this workload.
The redesign separates document processing from live search reads
CobbleDB is only one component of the new architecture. Perplexity also created Pillar for durable document state and publication, and Lorry for partition-aligned batch delivery.
Pillar is built on YTsaurus and stores components such as metadata, chunks and embeddings in separate table families. It tracks versions so multiple chunking or embedding representations can coexist while the underlying models evolve.
Pillar also decides which processed documents should be published downstream. Perplexity says it can define subsets such as fresh pages or high-value pages, allowing the more expensive NVMe-backed hot store to contain only the documents the search system actually wants available for serving.
Lorry then takes export records from a durable queue, groups them into partition-specific batch files and hands those batches to CobbleDB. This decouples large processing jobs from the latency-sensitive read path.
Re-embedding the web no longer has to compete directly with search traffic
The old architecture coupled document processing to direct writes into DynamoDB. A large reprocessing job could therefore create a wave of individual updates against the same hot store serving live search traffic.
That becomes a problem when Perplexity wants to change its chunking method, replace an embedding model or add a new field across a large corpus. Infrastructure maintenance can compete with user queries for throughput.
The new design separates durable processing state from hot-store ingestion. Pillar can track the updated document state, Lorry can batch the changes, and CobbleDB replicas can ingest them asynchronously at their own pace.
Perplexity says this lets it perform incremental updates or large rebuilds without forcing maintenance work onto the live query path.
Faster passage retrieval can reduce one component of answer latency
For users, CobbleDB is invisible. They do not query it directly, and a faster database does not automatically mean the entire Perplexity answer becomes five times faster.
AI answer latency includes many stages: interpreting the query, retrieving candidates, ranking them, loading prepared source content, reasoning with the evidence and generating the final response. CobbleDB accelerates one specific storage stage inside that pipeline.
Still, reducing median batch-read latency from 31.4 ms to 5.60 ms removes roughly 26 milliseconds from each measured batch operation. Lower p99 latency can be even more valuable because slow storage reads can hold up an otherwise fast retrieval pipeline.
At high query volumes and across multi-step agentic searches, small infrastructure improvements can compound.
The system beneath citations affects how quickly evidence reaches the model
AI citations are often discussed as a content-selection problem: which domains rank, which pages are retrieved and which passages support the answer. CobbleDB highlights another layer that is less visible but operationally essential.
Once a retrieval system has identified candidate pages, the answer engine still needs to fetch the processed content quickly enough to use it. The source may already have been crawled and embedded, but those representations must be served to the model during the query.
CobbleDB stores exactly that prepared representation. It therefore sits between source selection and model consumption.
Faster retrieval does not make a weak source authoritative or fix an incorrect citation. It makes the evidence that Perplexity’s retrieval and ranking systems have already selected available to the downstream model with less storage delay.
Perplexity estimates at least 20% lower storage cost
Latency was not the only motivation. Perplexity says its internal cost model estimates that CobbleDB saves at least 20% compared with DynamoDB for the workload.
The company attributes part of the difference to control over storage media and the ability to keep durable state on cheaper HDD-backed infrastructure while reserving NVMe capacity for the hot subset of documents used in search results.
Managed-service pricing also charges according to read and write usage, which becomes significant when large prepared records are repeatedly fetched at high query volume and entire corpora need to be reprocessed.
The 20% figure is an internal Perplexity estimate rather than an independently audited cost comparison, and the economics will not necessarily transfer to other organizations with different workloads or engineering costs.
Building instead of buying requires enough scale to justify specialization
CobbleDB illustrates a classic infrastructure tradeoff. General-purpose managed databases reduce operational complexity and work well across many workloads. Purpose-built systems can outperform them when an organization has enough scale and a sufficiently narrow access pattern to justify specialization.
Perplexity’s workload is unusually specific: repeated low-latency batch reads of prepared web-page records, backed by a separate asynchronous processing pipeline.
That specificity lets the company remove features it does not need and optimize aggressively around partition placement, caching, NVMe reads, replica selection and batch retrieval.
Most AI applications do not operate a web-scale search corpus or sustain hundreds of thousands of storage requests per second, so CobbleDB’s architecture should not be read as evidence that every AI company should replace managed databases with custom infrastructure.
Hundreds of coding agents helped two engineers build the core system
The report also offers a glimpse into how Perplexity is building infrastructure internally. The company says the core CobbleDB infrastructure was developed over two months by two human engineers working with hundreds of internal coding agents.
Perplexity says engineers remained responsible for architecture, consequential code review and authorization of production operations, while agents handled much of the continuous inspection and follow-up work between those decisions.
This is relevant because custom distributed databases have traditionally required large engineering investments. Perplexity argues that powerful coding agents shift the build-versus-buy calculation by reducing some of the human labor required to create specialized infrastructure.
That claim is specific to Perplexity’s experience, but it suggests that AI coding tools may affect not only application development but also the economics of building deep infrastructure in-house.
CobbleDB is planned for open source
Perplexity says it plans to open-source CobbleDB soon so other teams building AI-native search systems can use the same storage layer.
The wording is forward-looking: the report announces the intention to release the system, not that an open-source repository is already generally available. Developers should therefore distinguish the current production deployment inside Perplexity from the planned public release.
If released, the project could be most relevant to teams with retrieval workloads resembling Perplexity’s own: large prepared document corpora, batch key reads, asynchronous updates and a strong need to minimize both median and tail latency.
The surrounding Pillar and Lorry architecture is equally important conceptually, because much of the reported benefit comes from separating persistent document state, update delivery and query-time serving rather than treating the database as an isolated component.
AI search competition is moving deeper into the infrastructure stack
Perplexity’s CobbleDB project shows that competition between answer engines is not limited to frontier language models. Search quality and responsiveness depend on crawling, chunking, embeddings, ranking, storage, caching and serving systems that users never see.
A model can only reason over evidence that the retrieval stack can find and deliver. At Perplexity’s scale, even the database holding prepared passages and embeddings becomes a product-performance lever.
The reported production migration reduced median batch-read latency from 31.4 to 5.60 milliseconds and p99 from 123 to 24.2 milliseconds at roughly 200,000 requests per second. Those are storage-layer measurements, not a claim that full AI answers are five times faster, and the company correctly labels the production comparison as observational before-and-after data.
But the direction is clear. Perplexity is optimizing not only which sources its AI search retrieves, but how quickly the underlying passages can move from storage into the model’s context. The infrastructure beneath AI citations is becoming a competitive layer of its own.