One-sentence Summary

Microsoft Research proposes Garnet, a next-generation cache-store that adopts the Redis wire protocol for compatibility while redesigning the storage, network, and cluster stack from a database perspective, achieving up to $100\times$100× higher throughput and $4\times$4× lower latency at high percentiles for real-world applications and services.

Key Contributions

• Garnet is a Redis-wire-protocol-compatible cache-store that rethinks cache-store design from a database perspective, using a hybrid shared-nothing session design and shared-everything storage engine to express hundreds of Redis commands over complex heap objects with transaction support and larger-than-memory, memory-optimized operation.

• The Tsavorite storage engine provides a latch-free concurrent larger-than-memory, memory-optimized design with a narrow-waist interface, optimistic watching for transactions, and epoch-based slot revivification; a tunable checkpointing approach adapted from the CPR protocol enables consistent checkpoints without a write-ahead log and allows deployment as either a cache or a database with varying data-loss tolerance.

• Garnet’s cluster design uses sharding, replication, and migration protocols based on epoch-protected state machines to serve as a drop-in replacement for RESP cache-stores, delivering up to 100 times higher throughput and four times lower latency at high percentiles, supporting lower end-to-end costs and wide production use at Microsoft with open-source availability.

Introduction

Cache-stores such as Redis, Valkey, and Dragonfly have become central to modern services, supporting low-latency caching, session state, rate limiting, leaderboards, queues, durable storage, and other data-intensive workloads through rich remote protocols and complex data types. Prior systems are largely built around string-oriented get/set designs, single-threaded or per-core shared-nothing architectures, and in-memory hash tables, which can limit throughput, larger-than-memory scalability, checkpointing, heap-object support, and tunable durability on modern multi-core servers. The authors introduce Garnet, a RESP-compatible next-generation cache-store, and the authors leverage a latch-free tiered storage engine called Tsavorite, a pluggable network layer, deterministic durability mechanisms, and epoch-protected cluster abstractions. They express a large RESP command set over a narrow storage interface, aiming to provide drop-in compatibility while achieving bare-metal performance, flexible durability, and much higher throughput than existing cache-stores.

Method

The authors design Garnet as a single-node cache-store system that connects remote RESP clients to a high-performance storage engine through a network layer, a command parsing layer, and a shared storage substrate. Refer to the framework diagram.

The network layer exposes endpoints such as TCP sockets, Unix domain sockets, or user-defined endpoints, including support for low-latency networking stacks. Remote clients establish sessions with these endpoints, optionally performing a TLS handshake when encrypted communication is required. Each session is strictly FIFO, meaning that the server receives a stream of request commands and produces a pipelined stream of responses. Network buffers within a session are processed by the thread that receives them, such as a thread from the operating system pool or a thread selected by a kernel bypass library. That thread can handle TLS decryption and then passes the packet to the session parsing and query processing logic.

Incoming packets contain batches of RESP commands. Garnet first parses each command with an optimized predictive parser. The parser is designed to favor common commands such as GET and SET and to reduce branch mispredictions by exploiting the predictable structure of RESP syntax. After parsing, the command and its arguments are routed to the corresponding command processor. This design allows a batch of commands carried by a single packet to be processed without unnecessary thread switching, improving cache locality and reducing latency.

Storage operations are handled through Tsavorite, the key-value storage layer used by Garnet. Garnet creates an instance of Tsavorite at startup and registers the supported command set. Each Garnet session holds its own Tsavorite session, while the underlying data store is shared across sessions. This architecture keeps the layers above Tsavorite, such as TLS processing and parsing, thread-safe and task-parallel by construction. Multiple shared-nothing sessions operate on one shared-everything data store, giving the system session isolation without sacrificing performance or scalability.

The central storage interface is RUMDS, which stands for Read, Upsert, Modify, Delete, and Scan. Garnet’s query processor maps RESP commands to RUMDS calls and passes command parameters as opaque inputs. This abstraction allows Garnet to express a large set of existing RESP data commands while still being general enough to support new command classes. Read invokes a Reader callback to transform the stored value into the network response. Upsert can create or overwrite a value and uses callbacks for in-place or initial record writing. Modify supports atomic read-modify-write behavior through callbacks that decide whether an operation should create an initial value, update an existing in-memory value in place, or copy and update a record after reading older state. Delete inserts a tombstone or marks an existing record as deleted. Scan traverses live records for operations such as DBSIZE and SCAN.

Durability is implemented by writing deterministic operations to a Deterministic Operation Log, or DOL. When durability is enabled, commands that modify the store generate DOL entries. The DOL records are logged at the RUMDS interface rather than as raw RESP commands. Each entry contains enough information for deterministic replay, including the encoded command, parameters, database version, and nondeterministic inputs such as timestamps or random values. The DOL can be written to local disk or tiered storage. Commits can be issued periodically or after individual operations, and write-through or acknowledgment behavior can be tuned according to the desired durability guarantee.

Tsavorite itself uses a dual-store layout. A string store holds byte sequences, including data types that can be represented as byte arrays, while an object store holds heap objects such as complex data structures. Keys are strings in both stores. Each store consists of a latch-free concurrent hash index over a record log that spans main memory and storage tiers. The hash index points to logical addresses in the record log, and records are organized into fixed-size pages. The string store keeps key and value bytes inline within records, whereas the object store keeps in-memory pointers to heap objects. When pages are flushed or evicted, object values are serialized into byte representations for disk storage. Tiering is implemented through an IDevice abstraction, where a logical device can coordinate multiple real storage levels. Reads are served by the highest level that contains the data, and writes can be propagated through multiple tiers.

The concurrency model is supported by an epoch-protected state machine, called EPSM. EPSM is used for control-plane operations that must occur concurrently with normal data-plane work, including hash table growth, checkpointing, safe page flushing, eviction, and key migration. EPSM starts in a REST phase and moves through a sequence of phases. Custom actions can be registered to execute before a phase change. After a phase change, EPSM advances a global epoch counter from $e$e to $e+1$e+1. Threads performing store operations enter the current epoch, read the phase, and operate under the assumption that the phase is stable until the operation completes and the thread leaves the epoch. Once all threads have advanced beyond the old epoch, EPSM can perform the next phase action safely. This mechanism allows control-plane work to proceed without forcing every application thread to stop. The implementation avoids latency spikes by using a background task that drives EPSM, rather than relying on unlucky user threads to perform the work.

EPSM phase transitions also support important system behaviors. Hash table resizing can allocate a new table, synchronize threads through a barrier phase, move records to the new table, and return to REST after freeing the old table. Checkpointing can move the database from version $v$v to $v+1$v+1, allow old-version transactions to drain while newer operations proceed, and then flush record log pages to obtain a consistent snapshot. Slot migration can track batches of keys using a compact Bloom filter and transition through queued, migrating, deleting, and migrated states. These phases coordinate concurrent reads, writes, and transactions without requiring coarse slot-level locks. Representative phase sequences, along with record header and DOL layouts used by the storage and logging mechanisms, are explained as follows.

Record entries begin with a compact header that stores metadata such as the logical address of the previous record in a hash chain, tombstone status, dirty status, read cache status, and the existence of optional fields. This is followed by the key bytes and value bytes, with optional expiration timestamps and entity tags. For the object store, in-memory values are represented by pointers to heap objects, while on-disk values are serialized byte sequences. The DOL entry format similarly stores a compact record containing operation type, store version, session information, database identifier, and operation input. These layouts are designed to minimize space overhead and allow efficient recovery, replication, and concurrency control.

Tsavorite implements concurrent RUMDS operations using latch-free structural modifications combined with application-level bucket locks for record access. Reads follow the hash chain, starting from memory and moving to disk if necessary, and use full key comparison to locate the correct record. Upsert and Modify operations first search the mutable in-memory region. If the target record is present, updates may be applied in place using in-place writer or updater callbacks. If the operation requires a new record, Tsavorite allocates space at the tail of the log using atomic fetch-and-add, constructs the record through callbacks, and attempts to install the updated hash chain entry using compare-and-swap. On compare-and-swap failure, the allocated space is marked for reuse and the operation is retried. Additional post-update callbacks can be used for work that should occur only after a successful structural change, such as writing to the DOL.

Memory efficiency is handled through revivification and read caching. Expired or deleted records can occupy wasted space in memory, especially when in-place updates fail or log growth occurs. Revivification maintains a free list of reusable record slots. When records become safe to elide from hash chains, they are added to the free list, and future writes can reuse slots of appropriate size. A read cache can also be placed between the hash index and the main record log to retain frequently accessed records in memory, reducing expensive disk accesses for hot workloads.

Multi-key transactions are supported by combining pessimistic two-phase locking with optimistic watching. For standard RESP transactions, server-side scripts, stored procedures, and cross-key commands, Garnet first identifies the read and write sets from queued commands. It then acquires shared or exclusive locks on the corresponding hash buckets in sorted order to prevent deadlocks. Before lock acquisition, clients can optionally watch keys by recording version numbers from a small shared watch table. After locks are acquired, Garnet verifies whether any watched key has been modified since the watch was issued. If a conflict is detected, the transaction is aborted. This hybrid model reduces blocking for workloads dominated by single-key operations while still providing correct transaction semantics.

The checkpoint mechanism avoids the expensive process-fork approach used by some earlier systems. Garnet implements a non-blocking snapshot mechanism inspired by concurrent prefix recovery. EPSM coordinates the transition from an old database version $v$v to a new version $v+1$v+1. During the drain phase, transactions from either version may execute concurrently. The decision to assign a transaction to $v$v or $v+1$v+1 can be made after locks are acquired, which reduces the need to stall active transactions. Once all $v$v transactions complete, the system flushes $v$v records to obtain a prefix-consistent snapshot. This design also extends across the string store and object store by decoupling the state machine from the individual stores, allowing both stores to move through checkpoint phases under a common version transition.

In cluster mode, Garnet partitions the key space into hash slots and assigns slots to primary nodes. Clients access data through any node, and each node uses slot assignment information to serve requests or redirect clients to the correct owner. Primary nodes own subsets of slots and serve read and write requests, while secondary nodes attach to primaries and serve read requests. Cluster configuration changes propagate through a gossip protocol. During scale-out, slot migration can be performed using server-to-server communication rather than client-driven migration. Key sets are migrated in small batches, and EPSM coordinates concurrent access so that operations can continue when safe while preventing inconsistent reads or writes during critical transfer and deletion phases.

Replication uses the DOL as the synchronization mechanism between primaries and secondaries. A secondary can begin full or partial synchronization depending on whether the primary still has the required DOL address space. If a full snapshot is required, the primary can transmit a checkpoint to the secondary. Once recovery completes, the primary streams DOL records to the secondary. The secondary stores these records locally and asynchronously applies Upsert and Modify operations to its Tsavorite instance. Since DOL entries include database versions, the secondary can skip records already included in a recovered checkpoint, ensuring that each operation is applied exactly once. A streaming snapshot variant allows the primary to produce a consistent point-in-time snapshot by scanning immutable and then mutable record log regions without blocking normal operations or forcing disk writes.

Experiment

The experiments evaluate Garnet against Valkey, KeyDB, and Dragonfly under standalone and cluster workloads on Azure VMs using standard RESP benchmarks and YCSB-inspired traffic. Results show that Garnet delivers substantially higher throughput, lower and more stable latency, and better scalability for key-value, batched, large-payload, and complex-data operations, while its chosen architecture outperforms lock-based, worker-queue, and process-sharded alternatives. In durability and cluster settings, Garnet checkpoints near the available storage write limit and completes slot migration and replication synchronization faster with greater throughput stability than the competing systems.

The comparison shows Garnet provides the highest normalized node throughput and lowest latency among the compared cache stores while also offering a broader feature set. Its advantage is especially visible in checkpointing, cluster migration, and replication, where it sustains higher throughput and lower tail latency than the other systems under growing secondary counts. Garnet has much higher throughput than Redis, Valkey, KeyDB, and Dragonfly Garnet delivers lower batched latency than Redis, Valkey, KeyDB, and Dragonfly Garnet uniquely combines kernel bypass, tiered storage, decoupled storage, and tunable durability Garnet checkpoints data faster than Valkey, KeyDB, and Dragonfly Garnet migrates cluster slots faster than Valkey and Dragonfly Garnet keeps higher write throughput during replication with multiple secondaries

The RESP examples show basic operations on raw strings and lists, including reads, assignments, conditional writes, and values with expiration. They also show a cross-key list operation that moves an element from one key to another. These command patterns support multi-key transactions and atomic cross-key behavior in the storage layer. Raw string commands cover common cache operations such as reading values, setting values, writing only when a condition is met, and assigning values with an expiration time. List commands include moving an element between source and destination keys, illustrating a transactional operation across multiple key instances.

The mapping shows that RESP commands can be expressed using a small set of RUMDS operations and application callbacks. GET, SET, and SET-if-exists rely on Read, Upsert, or Modify operations, with callbacks handling response formatting, initial writes, and conditional updates. Related evaluation reports favorable checkpointing, migration, and replication performance for Garnet compared with other systems. Command handling is split into front-end operations and storage callbacks. Different RESP commands can share the same storage interface through tailored callbacks. The evaluated Garnet implementation sustains strong performance under checkpointing, migration, and replication workloads.

Experiments evaluate Garnet against Redis, Valkey, KeyDB, and Dragonfly across basic throughput and latency behavior and secondary-aware workloads such as checkpointing, cluster migration, and replication. The qualitative findings show Garnet consistently delivering higher normalized node throughput, lower batched latency, and stronger tail-latency behavior, especially as the number of secondaries grows. Additional command-mapping evaluations show common RESP string and list operations can be expressed through a small set of storage operations and callbacks, supporting conditional writes, expiration, and atomic cross-key list movements.