LevelDB

{{short description|Open-source key-value store by Google}}

{{Infobox software

| name = LevelDB

| title = LevelDB

| logo =

| screenshot =

| caption =

| collapsible =

| author =

| developer = Jeffrey Dean, Sanjay Ghemawat, Google Inc.

| released =

| discontinued =

| latest release version = {{wikidata|property|edit|reference|P348}}

| latest release date = {{start date and age|{{wikidata|qualifier|P348|P577}}}}

| latest preview version =

| latest preview date =

| programming language = C++

| operating system =

| platform =

| size = 350 kB (binary size)

| language =

| genre = Database library

| license = New BSD License

}}

LevelDB is an open-source on-disk key-value store written by Google fellows Jeffrey Dean and Sanjay Ghemawat.{{cite web |url=http://research.google.com/people/jeff/|title=Google Research Scientists and Engineers: Jeffrey Dean|publisher=Google, Inc}}{{cite web |url=http://research.google.com/people/sanjay/|title=Research Scientists and Engineers: Sanjay Ghemawat|publisher=Google, Inc}} Inspired by Bigtable,{{cite web | url=http://www.readwriteweb.com/hack/2011/07/google-open-sources-nosql-data.php | title=Google Open-Sources NoSQL Database Called LevelDB | work=ReadWriteWeb | date=July 30, 2011 | accessdate=July 30, 2011 | archive-url=https://web.archive.org/web/20110816070240/http://www.readwriteweb.com/hack/2011/07/google-open-sources-nosql-data.php | archive-date=August 16, 2011 | url-status=dead }} LevelDB source code is hosted on GitHub under the New BSD License and has been ported to a variety of Unix-based systems, macOS, Windows, and Android.{{cite web |url=http://google-opensource.blogspot.com/2011/07/leveldb-fast-persistent-key-value-store.html|title=Google Open Source Blog: LevelDB: A Fast Persistent Key-Value Store|publisher=Google, Inc}}

Features

LevelDB stores keys and values in arbitrary byte arrays, and data is sorted by key. It supports batching writes, forward and backward iteration, and compression of the data via Google's Snappy compression library.

LevelDB is not an SQL database. Like other NoSQL and dbm stores, it does not have a relational data model and it does not support SQL queries. Also, it has no support for indexes. Applications use LevelDB as a library, as it does not provide a server or command-line interface.

MariaDB 10.0 comes with a storage engine which allows users to query LevelDB tables from MariaDB.[https://mariadb.com/kb/en/mariadb/leveldb/ LevelDB storage engine]

History

LevelDB is based on concepts from Google's Bigtable database system. The table implementation for the Bigtable system was developed starting in about 2004, and is based on a different Google internal code base than the LevelDB code. That code base relies on a number of Google code libraries that are not themselves open sourced, so directly open sourcing that code would have been difficult. Jeff Dean and Sanjay Ghemawat wanted to create a system resembling the Bigtable tablet stack that had minimal dependencies and would be suitable for open sourcing, and also would be suitable for use in Chrome for the IndexedDB implementation. They wrote LevelDB starting in early 2011, with the same general design as the Bigtable tablet stack, but not sharing any of the code.{{cite web |url=http://groups.google.com/d/msg/leveldb/Dby98QYzJqw/s5elVIkpIAUJ|title=LevelDB mailing list: "Current Status of LevelDB"|author=Jeff Dean}}

Usage

LevelDB is used as the backend database for Google Chrome's IndexedDB and is one of the supported backends for Riak.[http://docs.basho.com/riak/latest/ops/advanced/backends/leveldb/ LevelDB]. Docs.basho.com. Retrieved on 2013-09-18. Additionally, Bitcoin Core and go-ethereum store the blockchain metadata using a LevelDB database.{{cite web |url=http://chimera.labs.oreilly.com/books/1234000001802/ch07.html?intcmp=il-na-books-videos-lp-bc15_20141218_radar_mastering_bitcoin_excerpt_chapter_seven|title=Chapter 7. The Blockchain|author=Andreas M. Antonopoulos|accessdate=8 January 2015}} Minecraft Bedrock Edition uses a modified version for chunk and entity data storage.{{cite web |url=https://minecraft.wiki/w/Bedrock_Edition_level_format |title=Bedrock Edition level format |work=Minecraft Wiki |access-date=24 September 2023 |language=en}} Autodesk AutoCAD 2016 also uses LevelDB.

Performance

Google has provided benchmarks comparing LevelDB's performance to SQLite and Kyoto Cabinet in different scenarios.{{cite web|url=http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html|title=LevelDB Benchmarks|publisher=Google, Inc.|url-status=dead|archiveurl=https://web.archive.org/web/20110820001028/http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html|archivedate=2011-08-20}} LevelDB outperforms both SQLite and Kyoto Cabinet in write operations and sequential-order read operations. LevelDB also excels at batch writes, but is slower than SQLite when dealing with large values. The currently published benchmarks were updated after SQLite configuration mistakes were noted in an earlier version of the results.{{cite web | url=http://sqlite.1065341.n5.nabble.com/LevelDB-benchmark-td62054.html | title=LevelDB Benchmark discussion}} Updated benchmarks[http://symas.com/mdb/microbench/ Database Microbenchmarks] {{webarchive|url=https://web.archive.org/web/20140809204443/http://symas.com/mdb/microbench/ |date=2014-08-09 }}, Symas Corp., 2012-09. Retrieved 22 October 2016 show that LevelDB also outperforms Berkeley DB, but these tests also show that OpenLDAP LightningDB is much faster (~10 times in some scenarios) in read operations and some write types (e.g. batch and synchronous writes, see the link above), and is almost equal in the rest of the test.

All the above benchmarks date back from 2011 to 2014, and may only be of historical significance as SQLite, for instance, became significantly more efficient.{{cite web|url=https://www.sqlite.org/cpu.html|title=Measuring and Reducing CPU Usage in SQLite }}

Bugs and reliability

LevelDB has a history of database corruption bugs.[https://docs.basho.com/riak/latest/ops/running/recovery/repairing-leveldb/ Repairing LevelDB][https://github.com/google/leveldb/issues?utf8=%E2%9C%93&q=corrupt+is%3Aissue Issues · google/leveldb · GitHub][https://bugs.chromium.org/p/chromium/issues/detail?id=261623 Unrecoverable corruption in Chromium][https://forum.syncthing.net/t/panic-leveldb-table-corruption-on-data-block/2526/23 Corruption in syncthing][https://github.com/google/leveldb/issues/333 Corruption after power loss][http://ethereum.stackexchange.com/questions/1159/corruption-on-data-block-while-synchronising Corruption in Ethereum] A study from 2014 has found that, on non-checksummed file systems, the database could become corrupted after a crash or power failure.{{cite book|url=https://www.usenix.org/conference/osdi14/technical-sessions/presentation/pillai|title=All File Systems Are Not Created Equal: On the Complexity of Crafting Crash-Consistent Applications|year=2014|pages=433–448|isbn=9781931971164}}

See also

{{Portal|Free and open-source software}}

References

{{Reflist}}