Key–value database

This article is about a storage system architecture. For the mode of data representation, see Key–value pair.

A key–value database, or key–value store, is a data storage paradigm designed for storing, retrieving, and managing associative arrays, and a data structure more commonly known today as a dictionary or hash table. Dictionaries contain a collection of objects, or records, which in turn have many different fields within them, each containing data. These records are stored and retrieved using a key that uniquely identifies the record, and is used to find the data within the database.

A tabular data card proposed for Babbage's Analytical Engine showing a key–value pair, in this instance a number and its base-ten logarithm

A table showing different formatted data values associated with different keys

Key–value databases work in a very different fashion from the better known relational databases (RDB). RDBs predefine the data structure in the database as a series of tables containing fields with well defined data types. Exposing the data types to the database program allows it to apply a number of optimizations. In contrast, key–value systems treat the data as a single opaque collection, which may have different fields for every record. This offers considerable flexibility and more closely follows modern concepts like object-oriented programming. Because optional values are not represented by placeholders or input parameters, as in most RDBs, key–value databases often use far less memory to store the same data, which can lead to large performance gains in certain workloads.

Performance, a lack of standardization and other issues have limited key–value systems to niche uses for many years, but the rapid move to cloud computing after 2010 has led to a renaissance as part of the broader NoSQL movement. Some graph databases, such as ArangoDB, are also key–value databases internally, adding the concept of the relationships (pointers) between records as a first class data type.

Types and examples

Key–value databases can use consistency models ranging from eventual consistency to serializability. Some support ordering of keys.

Some maintain data in memory (RAM), while others employ solid-state drives or rotating disks.

Every entity (record) is a set of key–value pairs. A key has multiple components, specified as an ordered list. The major key identifies the record and consists of the leading components of the key. The subsequent components are called minor keys. This organization is similar to a directory path specification in a file system (e.g., /Major/minor1/minor2/). The “value” part of the key–value pair is simply an uninterpreted string of bytes of arbitrary length.

The Unix system provides dbm (database manager), which is a 1979 library originally written by Ken Thompson. It is also ported to Microsoft Windows, provided through programming languages such as Perl for Win32. The dbm manages associative arrays of arbitrary data by use of a single key (a primary key). Modern implementations include sdbm, GNU dbm, and Berkeley DB. Although dbm precedes the concept of a NoSQL and is rarely mentioned in modern discourse, it is used by many pieces of software.

A more recent example of a key-value database is RocksDB which is used as storage engine for other database management systems such as ArangoDB. Other examples include Aerospike (database), Amazon DynamoDB, Memcached, Redis, and ScyllaDB.

See also

• Big data
• Data analysis
• Distributed data store
• Document-oriented database
• Multi-model database
• Tuple space
• Ordered Key-Value Store
• Name–value pair