TiDB
{{short description|NewSQL relational distributed database}}
{{Use mdy dates|date=February 2019}}
{{Infobox software
| name = TiDB
| developer = PingCAP Inc.
| released = {{Start date and age|2017|10|15}}{{cite web|title=1.0 GA release notes|website=GitHub |url=https://github.com/pingcap/tidb/releases/tag/v1.0.0}}
| latest release version = {{wikidata|property|edit|reference|P348}}
| latest release date = {{start date and age|{{wikidata|qualifier|P348|P577}}}}
| programming language = Go (TiDB), Rust (TiKV)
| language = English, Chinese
| genre = NewSQL
| license = Apache 2.0
}}
TiDB (/’taɪdiːbi:/, "Ti" stands for Titanium) is an open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads.{{cite web|last=Xu|first=Kevin|date=October 17, 2018|title=How TiDB combines OLTP and OLAP in a distributed database|url=https://www.infoworld.com/article/3313327/how-tidb-combines-oltp-and-olap-in-a-distributed-database.html|archive-url=|archive-date=|access-date=|website=InfoWorld}} Designed to be MySQL compatible, it is developed and supported primarily by PingCAP and licensed under Apache 2.0. It is also available as a paid product. TiDB drew its initial design inspiration from Google's Spanner and F1 papers.{{cite web |year=2013 |title=F1: A Distributed SQL Database That Scales |url=https://ai.google/research/pubs/pub41344}}{{cite web |year=2012 |title=Spanner: Google's Globally-Distributed Database |url=https://ai.google/research/pubs/pub39966}}{{cite web|last=Hall|first=Susan|date=April 17, 2017|title=TiDB Brings Distributed Scalability to SQL|url=https://thenewstack.io/tidb-brings-distributed-scalability-sql/|archive-url=|archive-date=|access-date=|website=The New Stack}}
Release history
See all [https://docs.pingcap.com/tidb/stable/release-notes TiDB release notes].
- On December 19, 2024, [https://docs.pingcap.com/tidb/stable/release-8.5.0 TiDB 8.5 GA] was released.
- On May 24, 2024, [https://docs.pingcap.com/tidb/stable/release-8.1.0 TiDB 8.1 GA] was released.
- On December 1, 2023, [https://www.pingcap.com/press-release/tidb-7-5-sets-new-standard-stability-scale/ TiDB 7.5 GA] was released.
- On May 31, 2023, [https://www.pingcap.com/blog/release-tidb-7-1-lts/ TiDB 7.1 GA] was released.
- On April 7, 2022, [https://en.pingcap.com/blog/tidb-6-0-a-leap-towards-an-enterprise-grade-cloud-database/ TiDB 6.0 GA] was released.
- On April 7, 2021 [https://docs.pingcap.com/tidb/stable/release-5.0.0 TiDB 5.0 GA] was released.
- On May 28, 2020, [https://docs.pingcap.com/tidb/stable/release-4.0-ga TiDB 4.0 GA] was released.
- On June 28, 2019, [https://docs.pingcap.com/tidb/stable/release-3.0-ga TiDB 3.0 GA] was released.
- On April 27, 2018, [https://docs.pingcap.com/tidb/stable/release-2.0-ga TiDB 2.0 GA] was released.
- On October 16, 2017, [https://docs.pingcap.com/tidb/stable/release-1.0-ga TiDB 1.0 GA] was released.
Main features
= Horizontal scalability =
TiDB can expand both SQL processing and storage capacity by adding new nodes.
= MySQL compatibility =
TiDB acts like it is a MySQL 8.0 server to applications. A user can continue to use all of the existing MySQL client libraries.{{cite web|last=Tocker|first=Morgan|date=November 14, 2018|title=Meet TiDB: An open source NewSQL database|url=https://opensource.com/article/18/11/key-differences-between-mysql-and-tidb|archive-url=|archive-date=|access-date=|website=Opensource.com}} Because TiDB's SQL processing layer is built from scratch, not a MySQL fork.{{cite web|last=|first=|date=|title=Compatibility with MySQL|url=https://docs.pingcap.com/tidb/stable/mysql-compatibility|archive-url=|archive-date=|access-date=|website=PingCAP}}
= Distributed transactions with strong consistency =
TiDB internally shards a table into small range-based chunks that are referred to as "Regions".{{cite web|last=|first=|date=|title=TiKV Architecture|url=https://tikv.org/docs/4.0/concepts/architecture/#region|archive-url=|archive-date=|access-date=|website=TiKV}} Each Region defaults to approximately 100 MB in size, and TiDB uses a two-phase commit internally to ensure that regions are maintained in a transactionally consistent way.
= Cloud native =
TiDB is designed to work in the cloud. The storage layer of TiDB, called TiKV, became a Cloud Native Computing Foundation (CNCF) member project in August 2018, as a Sandbox level project,{{cite web|last=Evans|first=Kristen|date=August 28, 2018|title=CNCF to Host TiKV in the Sandbox|url=https://www.cncf.io/blog/2018/08/28/cncf-to-host-tikv-in-the-sandbox/|archive-url=|archive-date=|access-date=|website=Cloud Native Computing Foundation}} and became an incubation-level hosted project in May 2019.{{Cite web|last=CNCF|date=May 21, 2019|title=TOC Votes to Move TiKV into CNCF Incubator|url=https://www.cncf.io/blog/2019/05/21/toc-votes-to-move-tikv-into-cncf-incubator/|archive-url=|archive-date=|access-date=August 19, 2020|website=Cloud Native Computing Foundation}} TiKV graduated from CNCF in September 2020.{{Cite web|last=TiKV Authors|date=September 2, 2020|title=Celebrating TiKV's CNCF Graduation|url=https://tikv.org/blog/graduation-announcement/|archive-url=|archive-date=|access-date=|website=TiKV}}
= Real-time HTAP =
= High availability =
TiDB uses the Raft consensus algorithm{{cite web|title=The Raft Consensus Algorithm |url=https://raft.github.io/}} to ensure that data is available and replicated throughout storage in Raft groups. In the event of failure, a Raft group will automatically elect a new leader for the failed member, and self-heal the TiDB cluster.
Deployment methods
= Kubernetes with Operator =
TiDB can be deployed in a Kubernetes-enabled cloud environment by using TiDB Operator.{{cite web|last=Jackson|first=Joab|date=January 22, 2019|title=Database Operators Bring Stateful Workloads to Kubernetes|url=https://thenewstack.io/databases-operators-bring-stateful-workloads-to-kubernetes/|archive-url=|archive-date=|access-date=|website=The New Stack}} An Operator is a method of packaging, deploying, and managing a Kubernetes application. It is designed for running stateful workloads and was first introduced by CoreOS in 2016.{{cite web|last=Philips|first=Brandon|date=November 3, 2016|title=Introducing Operators: Putting Operational Knowledge into Software|url=https://coreos.com/blog/introducing-operators.html|archive-url=|archive-date=|access-date=|website=CoreOS}} TiDB Operator{{cite web|last=|first=|date=|title=TiDB Operator GitHub repo|url=https://github.com/pingcap/tidb-operator|archive-url=|archive-date=|access-date=|website=GitHub}} was originally developed by PingCAP and open-sourced in August, 2018.{{cite web|last=|first=|date=August 16, 2018|title=Introducing the Kubernetes Operator for TiDB|url=https://www.infoworld.com/article/3297700/introducing-the-kubernetes-operator-for-tidb.html|archive-url=|archive-date=|access-date=|website=InfoWorld}} TiDB Operator can be used to deploy TiDB on a laptop,{{cite web|last=|first=|date=|title=Deploy TiDB to Kubernetes on Your Laptop|url=https://docs.pingcap.com/tidb-in-kubernetes/stable/get-started|archive-url=|archive-date=|access-date=|website=}} Google Cloud Platform’s Google Kubernetes Engine,{{cite web|last=|first=|date=|title=Deploy TiDB, a distributed MySQL compatible database, to Kubernetes on Google Cloud|url=https://docs.pingcap.com/tidb-in-kubernetes/stable/deploy-on-gcp-gke|archive-url=|archive-date=|access-date=|website=}} and Amazon Web Services’ Elastic Container Service for Kubernetes.{{cite web|title=Deploy TiDB, a distributed MySQL compatible database, on Kubernetes via AWS EKS |website=GitHub |url=https://github.com/pingcap/tidb-operator/blob/master/docs/aws-eks-tutorial.md}}
= TiUP =
TiDB 4.0 introduces [https://github.com/pingcap/tiup TiUP], a cluster operation and maintenance tool. It helps users quickly install and configure a TiDB cluster with a few commands.{{Cite web|last=Long|first=Heng|date=April 19, 2020|title=Get a TiDB Cluster Up in Only One Minute|url=https://pingcap.com/blog/get-tidb-cluster-up-in-only-one-minute|archive-url=|archive-date=|access-date=August 19, 2020|website=PinCAP}}
= TiDB Ansible =
TiDB can be deployed using Ansible by using a TiDB Ansible playbook (not recommended).{{cite web|title=Ansible Playbook for TiDB |website=GitHub |url=https://github.com/pingcap/tidb-ansible}}
= Docker =
Docker can be used to deploy TiDB in a containerized environment on multiple nodes and multiple machines, and Docker Compose can be used to deploy TiDB with a single command for testing purposes.{{cite web|title=How to Spin Up an HTAP Database in 5 Minutes With TiDB + TiSpark |url=https://dzone.com/articles/how-to-spin-up-an-htap-database-in-5-minutes-with}}
Tools
= TiDB Data Migration (DM) =
TiDB Data Migration (DM) is suited for replicating data from already sharded MySQL or MariaDB tables to TiDB.{{cite web|last=|first=|date=|title=DM GitHub Repo|url=https://github.com/pingcap/dm|archive-url=|archive-date=|access-date=|website=GitHub}} A common use case of DM is to connect MySQL or MariaDB tables to TiDB, treating TiDB almost as a slave, then directly run analytical workloads on this TiDB cluster in near real-time.
= Backup & Restore =
Backup & Restore (BR) is a distributed backup and restore tool for TiDB cluster data.{{Cite web|last=Shen|first=Taining|date=April 13, 2020|title=How to Back Up and Restore a 10-TB Cluster at 1+ GB/s|url=https://pingcap.com/blog/back-up-and-restore-a-10-tb-cluster-at-1-gb-per-second|archive-url=|archive-date=|access-date=|website=PingCAP}}
= Dumpling =
Dumpling is a data export tool that exports data stored in TiDB or MySQL. It lets users make logical full backups or full dumps from TiDB or MySQL.{{Cite web|last=|first=|date=|title=Dumpling Overview|url=https://docs.pingcap.com/tidb/stable/dumpling-overview|archive-url=|archive-date=|access-date=|website=PingCAP}}
= TiDB Lightning =
TiDB Lightning is a tool that supports high speed full-import of a large MySQL dump into a new TiDB cluster. This tool is used to populate an initially empty TiDB cluster with much data, in order to speed up testing or production migration. The import speed improvement is achieved by parsing SQL statements into key-value pairs, then directly generate Sorted String Table (SST) files to RocksDB.{{cite web|last=Chan|first=Kenny|date=January 30, 2019|title=Introducing TiDB Lightning|url=https://www.pingcap.com/blog/introducing-tidb-lightning/|archive-url=|archive-date=|access-date=|website=PingCAP}}{{Cite web|last=|first=|date=|title=TiDB Lightning Overview|url=https://docs.pingcap.com/tidb/stable/tidb-lightning-overview|archive-url=|archive-date=|access-date=|website=PingCAP}}
= TiCDC =
TiCDC is a change data capture tool which streams data from TiDB to other systems like Apache Kafka.
= TiDB Binlog =
TiDB Binlog is a tool used to collect the logical changes made to a TiDB cluster. It is used to provide incremental backup and replication, either between two TiDB clusters, or from a TiDB cluster to another downstream platform.{{Cite web|last=|first=|date=|title=TiDB Binlog Cluster Overview|url=https://docs.pingcap.com/tidb/stable/tidb-binlog-overview|archive-url=|archive-date=|access-date=|website=PingCAP}}{{cite web|last=Wang|first=Xiang|date=January 29, 2019|title=TiDB-Binlog Architecture Evolution and Implementation Principles|url=https://www.pingcap.com/blog/tidb-binlog-architecture-evolution-and-implementation-principles/|archive-url=|archive-date=|access-date=|website=PingCAP}}