kdb+

{{short description|High-speed database}}

{{lowercase title|kdb+}}

{{Infobox software

| name = kdb+

| logo =

| logo alt =

| screenshot =

| caption =

| screenshot alt =

| collapsible =

| author = Arthur Whitney

| developer = KX

| released = {{Start date and age|2003}}

| discontinued =

| latest release version =

| latest release date =

| latest preview version =

| latest preview date =

| programming language = q

| operating system = Windows, macOS, Linux, Solaris

| platform = IA-32, x86-64, SPARC

| size =

| language = English

| language count =

| language footnote =

| genre = relational time series database

| license = Proprietary

| alexa =

| website = {{URL|https://kx.com/}}

| standard =

| AsOf =

}}

kdb+ is a column-based relational time series database (TSDB) with in-memory (IMDB) abilities, developed and marketed by KX. The database is commonly used in high-frequency trading (HFT) to store, analyze, process, and retrieve large data sets at high speed. kdb+ has the ability to handle billions of records and analyzes data within a database. The database is available in 32-bit and 64-bit versions for several operating systems. Financial institutions use kdb+ to analyze time series data such as stock or commodity exchange data. The database has also been used for other time-sensitive data applications including commodity markets such as energy trading, telecommunications, sensor data, log data, machine and computer network usage monitoring{{cite journal |title=Kx Systems readies faster database in push beyond financial services |last=Stamper |first=Jason |date=September 4, 2014 |journal=451 Research}} along with real time analytics in Formula One racing.{{Cite web |title=Williams Racing Announce KX as an Official Supplier |url=https://www.williamsf1.com/posts/ea41af4f-c0b5-413c-a8d5-b7065586ff0e/williams-racing-announce-kx-as-an-official-supplier |access-date=2023-01-24 |website=Williams Racing |language=en}}

Overview

kdb+ is a high-performance column-store database that was designed to process and store large amounts of data. Commonly accessed data is pushed into random-access memory (RAM), which is faster to access than data in disk storage. Created with financial institutions in mind, the database was developed as a central repository to store time series data that supports real-time analysis of billions of records. kdb+ has the ability to analyze data over time and responds to queries similar to Structured Query Language (SQL).

Columnar databases return answers to some queries in a more efficient way than row-based database management systems. kdb+ dictionaries, tables and nanosecond time stamps are native data types and are used to store time series data.

At the core of kdb+ is the built-in programming language, q, a concise, expressive query array language, and dialect of the language APL. Q can manipulate streaming, real-time, and historical data. kdb+ uses q to aggregate and analyze data, perform statistical functions, and join data sets and supports SQL queries The vector language q was built for speed and expressiveness and eliminates most need for looping structures. kdb+ includes interfaces in C, C++, Java, C#, and Python.

History

In 1998, KX released kdb, a database built on the language K written by Arthur Whitney. In 2003, kdb+ was released as a 64-bit version of kdb. In 2004, the kdb+ tick market database framework was released along with kdb+ taq, a loader for the New York Stock Exchange (NYSE) taq data. kdb+ was created by Arthur Whitney, building on his prior work with array languages.

In April 2007, KX announced that it was releasing a version of kdb+ for Mac OS X. At that time, kdb+ was also available on the operating systems Linux, Windows, and Solaris.

In September 2012, version 3.0 was released. It was optimized for Intel's upgraded processors with support for WebSockets, Globally unique identifiers (GUID)s, and Universally unique identifiers (UUID). Intel's Advanced Vector Extensions (AVX) and Streaming SIMD Extensions 4 (SSE4) 4.2 on the Sandy Bridge processors of the time allowed for enhanced support of the kdb+ system. In June 2013, version 3.1 was released, with benchmarks up to 8 times faster than for older versions.

In March 2020, version 4.0 was released. New features included Multithreaded primitives, Intel Optane DC persistent memory support and Data at Rest Encryption.

References

{{reflist|35em| refs=

{{cite web |url=http://itl-g.com/solutions/kdb/ |title=Kdb+ |author= |publisher=Itl Group |access-date=June 8, 2016}}

{{cite web |url=http://insidebigdata.com/2015/01/19/kdb-internet-thingsbig-data/ |title=Kdb+ and the Internet of Things/Big Data |last=Gutierrez |first=Daniel |date=January 19, 2015 |publisher=Inside Big Data |access-date=June 8, 2016}}

{{cite web |url=http://www.financemagnates.com/institutional-forex/technology/first-derivatives-boosts-its-kdb-software-business-with-new-acquisition/ |title=First Derivatives Boosts its KDB+ Software Business with New Acquisition |last=Golovtchenko |first=Victor |date=January 1, 2016 |publisher=Finance Magnates |access-date=June 8, 2016}}

{{cite web |url=http://www.thetradingmesh.com/pg/blog/mike/read/5388/kx-systems-and-high-frequency-trading |title=Kx Systems and High Frequency Trading |author= |date=June 10, 2011 |publisher=The Trading Mesh |access-date=June 8, 2016 |archive-url=https://web.archive.org/web/20181219134322/http://www.thetradingmesh.com/pg/blog/mike/read/5388/kx-systems-and-high-frequency-trading |archive-date=December 19, 2018 |url-status=dead }}

{{cite web |url=http://www.automatedtrader.net/news/at/142863/kxs-kdb+-claims-up-to-8x-faster-than-any-previous-stac-m3--benchmarks |title=Kx's kdb+ claims up to 8x faster than any previous STAC M3 benchmarks |author= |date=June 11, 2013 |publisher=Automated Trader |access-date=June 8, 2016 |archive-url=https://web.archive.org/web/20181219090914/http://www.automatedtrader.net/news/at/142863/kxs-kdb+-claims-up-to-8x-faster-than-any-previous-stac-m3--benchmarks |archive-date=December 19, 2018 |url-status=dead }}

{{cite web |url=http://marketsmedia.com/wall-street-firms-extend-big-data-capabilities/ |title=Wall Street Firms Extend 'Big Data; Capabilities |last=Marlin |first=Steve |date=June 19, 2014 |publisher=Markets Media |access-date=June 8, 2016}}

{{cite web |url=http://quantmind.com/blog/2009/10/a-first-look-at-kdb |title=A first look at kdb+ |last=Sbardella |first=Luca |date=October 2, 2009 |publisher=Quantmind |access-date=June 8, 2016}}

{{cite web |url=http://www.clustermonkey.net/Deep-Dive/working-down-the-column-the-kdb-community.html |title=Working Down the Column: The Kdb+ Community |last=Eadline |first=Douglas |date=September 9, 2014 |publisher=Cluster Monkey |access-date=June 8, 2016}}

{{cite web |url=https://www.forbes.com/sites/tomgroenfeldt/2012/09/24/at-kx-systems-big-data-is-old-news/#1b2e873e7010 |title=At Kx Systems Big Data Is Old News |last=Groenfeldt |first=Tom |date=September 24, 2012 |work=Forbes |access-date=June 8, 2016}}

{{cite web |url=http://www.macworld.com/article/1057391/kx.html |title=Kdb+ financial database comes to Mac OS X |last=Cohen |first=Peter |date=April 17, 2007 |publisher=Macworld |access-date=June 8, 2016}}

{{cite web |url=http://www.wallstreetandtech.com/latency/kx-offers-nanosecond-timestamps-in-updated-database/d/d-id/1262409 |title=Kx Offers Nanosecond Timestamps in Updated Database |last=Crosman |first=Penny |date=September 14, 2009 |publisher=Information Week |access-date=June 8, 2016 |archive-date=August 16, 2017 |archive-url=https://web.archive.org/web/20170816234948/http://www.wallstreetandtech.com/latency/kx-offers-nanosecond-timestamps-in-updated-database/d/d-id/1262409 |url-status=dead }}

{{cite web |url=https://kx.com/blog/kdb-version-4-0-faster-more-secure/ | title=Kdb+ version 4.0 – faster, more secure |last=Garland | first=Simon |date=March 23, 2020 |access-date=December 4, 2021}}

}}