Elasticsearch

{{short description|Search engine}}

{{Multiple issues|

{{Primary sources|date=September 2015}}

{{Unreliable sources|date=September 2015}}}}

{{Use dmy dates|date=February 2019}}

{{Infobox software

| name = Elasticsearch

| logo = Elasticsearch logo.svg

| logo size = 240px

| screenshot =

| caption =

| developer = Elastic NV

| author = Shay Banon

| released = {{Start date and age|2010|02|08|df=y}}

| latest release version = {{Multiple releases

| branch1 = 8.x

| version1 = 8.15.0

| date1 = {{Start date and age|df=yes|2024|08|02}}{{cite web

| url = https://github.com/elastic/elasticsearch/releases

| title = Releases · elastic/elasticsearch

| via = GitHub

| access-date = 25 August 2023}}

| branch2 = 7.x

| version2 = 7.17.21

| date2 = {{Start date and age|df=yes|2024|05|02}}}}

| latest release date =

| operating system = Cross-platform

| repo = {{URL|https://github.com/elastic/elasticsearch}}

| programming language = Java

| genre = Search and index

| license = Triple-licensed Elastic License (proprietary; source-available), Server Side Public License (proprietary; source-available) and Affero General Public License (free and open-source)

}}

Elasticsearch is a search engine based on Apache Lucene, a free and open-source search engine. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Official clients are available in Java,{{cite web|url=https://github.com/elastic/elasticsearch-java/|title=Elasticsearch Java Client|website=github.com|access-date=2022-10-07}} .NET{{cite web|url=https://github.com/elastic/elasticsearch-net/|title=Elasticsearch .NET Client|website=github.com|access-date=2022-10-07}} (C#), PHP,{{cite web|url=https://github.com/elastic/elasticsearch-php/|title=Elasticsearch PHP Client|website=github.com|access-date=2022-10-07}} Python,{{cite web|url=https://github.com/elastic/elasticsearch-py/|title=Elasticsearch Python Client|website=github.com|access-date=2022-10-07}} Ruby{{cite web|url=https://github.com/elastic/elasticsearch-ruby/|title=Elasticsearch Ruby Client|website=github.com|access-date=2022-10-07}} and many other languages.{{cite web |title=Programming Language Clients |url=https://www.elastic.co/guide/en/enterprise-search/master/programming-language-clients.html |archive-url=https://web.archive.org/web/20230402031738/https://www.elastic.co/guide/en/enterprise-search/master/programming-language-clients.html |archive-date=2 April 2023 |access-date=2022-10-07 |website=elastic.co}} According to the DB-Engines ranking, Elasticsearch is the most popular enterprise search engine.{{cite web

| url = http://db-engines.com/en/ranking/search+engine

| title = DB-Engines Ranking - popularity ranking of search engines

| work = db-engines.com|access-date= 10 January 2016}}

History

Shay Banon created the precursor to Elasticsearch, called Compass, in 2004.

{{cite web|last=Banon|first=Shay|title=The Future of Compass & ElasticSearch|date=7 July 2010 |url=http://thedudeabides.com/articles/the_future_of_compass}} While thinking about the third version of Compass he realized that it would be necessary to rewrite big parts of Compass to "create a scalable search solution". So he created "a solution built from the ground up to be distributed" and used a common interface, JSON over HTTP, suitable for programming languages other than Java as well. Shay Banon released the first version of Elasticsearch in February 2010.{{cite web|last=Banon|first=Shay|title=You Know, for Search|url=http://www.elasticsearch.org/blog/2010/02/08/youknowforsearch.html|archive-url=https://web.archive.org/web/20130116045454/http://www.elasticsearch.org/blog/2010/02/08/youknowforsearch.html|archive-date=2013-01-16|date=2010-02-08}}

Elastic NV was founded in 2012 to provide commercial services and products around Elasticsearch and related software.{{cite web|url=https://www.elastic.co/about|title=Immediate Insight from Data Matters |work=elastic.co|access-date=25 March 2015}} In June 2014, the company announced raising $70 million in a Series C funding round, just 18 months after forming the company. The round was led by New Enterprise Associates (NEA). Additional funders include Benchmark Capital and Index Ventures. This round brought total funding to $104M.{{cite web |last=Miller |first=Ron |url=https://techcrunch.com/2014/06/05/elasticsearch-scores-70m-in-series-c-to-fund-growth-spurt/|title=ElasticSearch Scores $70M In Series C To Fund Growth Spurt|publisher=AOL|work=TechCrunch|date=5 June 2014 |access-date=4 April 2024}}

In March 2015, the company Elasticsearch changed its name to Elastic.{{cite web|title=Elasticsearch Changes Name to Elastic to Reflect Wide Adoption Beyond Search|url=https://www.elastic.co/about/press/elasticsearch-changes-name-to-elastic-to-reflect-wide-adoption-beyond-search|website=Elastic.co|date=10 March 2015|access-date=19 October 2016}}

In June 2018, Elastic filed for an initial public offering with an estimated valuation of between 1.5 and 3 billion dollars.{{Cite news|url=https://www.recode.net/2018/6/21/17491102/elastic-elasticsearch-ipo-confidential-filing-wall-street|title=The IPOs keep coming: The search company Elastic has filed to go public|last=Schleifer|first=Theodore|date=21 June 2018|work=Recode|access-date=22 June 2018|archive-url=https://web.archive.org/web/20181005131020/https://www.recode.net/2018/6/21/17491102/elastic-elasticsearch-ipo-confidential-filing-wall-street|archive-date=5 October 2018|url-status=dead}} On 5 October 2018, Elastic was listed on the New York Stock Exchange.{{cite web|url=https://www.elastic.co/blog/ze-bell-has-rung-thank-you-users-customers-and-partners|title=Ze Bell Has Rung: Thank You Users, Customers, and Partners|last=Banon|first=Shay|publisher=Elastic (NV)|access-date=24 October 2018|date=5 October 2018}}

Developed from the Found acquisition by Elastic in 2015,{{Cite web |last=Oliver |first=Andrew C. |date=2015-03-10 |title=Elasticsearch buys into search as a service, rebrands as 'Elastic' |url=https://www.infoworld.com/article/2894955/elasticsearch-buys-into-search-as-service-rebrands-as-elastic.html |access-date=2019-04-01 |website=InfoWorld.com |language=en}} Elastic Cloud is a family of Elasticsearch-powered SaaS offerings which include the Elasticsearch Service, as well as Elastic App Search Service, and Elastic Site Search Service which were developed from Elastic's acquisition of Swiftype.{{cite web |last=Ha |first=Anthony |date=9 November 2017 |title=Elastic acquires search startup Swiftype |url=https://techcrunch.com/2017/11/09/elastic-acquires-swiftype/ |access-date=3 April 2024 |website=TechCrunch |language=en-US}} In late 2017, Elastic formed partnerships with Google to offer Elastic Cloud in Google Cloud Platform (GCP), and Alibaba to offer Elasticsearch and Kibana in Alibaba Cloud.

Elasticsearch Service users can create secure deployments with partners, Google Cloud Platform (GCP) and Alibaba Cloud.{{Cite web |last=Yegulalp |first=Serdar |date=2017-04-07 |title=Google Cloud to host open source Elasticsearch |url=https://www.infoworld.com/article/3188332/google-cloud-to-host-open-source-elasticsearch.html |access-date=2019-04-01 |website=InfoWorld.com |language=en}}

= Licensing changes =

In January 2021, Elastic announced that starting with version 7.11, they would be relicensing their Apache 2.0 licensed code in Elasticsearch and Kibana to be dual licensed under Server Side Public License and the Elastic License, neither of which is recognized as an open-source license.{{cite web |last1=Banon |first1=Shay |title=Doubling down on open, Part II |url=https://www.elastic.co/blog/licensing-change |website=Elastic |date=14 January 2021 |access-date=19 January 2021}}{{Cite web|last=Vaughan-Nichols|first=Steven J.|title=Elastic changes open-source license to monetize cloud-service use|url=https://www.zdnet.com/article/elastic-changes-open-source-license-to-monetize-cloud-service-use/|access-date=2021-01-23|website=ZDNet|language=en}} Elastic blamed Amazon Web Services (AWS) for this change, objecting to AWS offering Elasticsearch and Kibana as a service directly to consumers and claiming that AWS was not appropriately collaborating with Elastic.{{cite web |last1=Banon |first1=Shay |title=Amazon: NOT OK - why we had to change Elastic licensing |url=https://www.elastic.co/blog/why-license-change-AWS |website=Elastic |date=19 January 2021 |access-date=19 January 2021}} Critics of the re-licensing decision predicted that it would harm Elastic's ecosystem and noted that Elastic had previously promised to "never....change the license of the Apache 2.0 code of Elasticsearch, Kibana, Beats, and Logstash". Amazon responded with plans to fork the projects and continue development under Apache License 2.0.{{Cite news |last1=Banon |first1=Shay |date=2024-08-29 |title=Elasticsearch is Open Source, Again |url=https://www.elastic.co/blog/elasticsearch-is-open-source-again}}{{Cite web|date=2021-01-21|title=Stepping up for a truly open source Elasticsearch|url=https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/|access-date=2021-01-28|website=Amazon Web Services|language=en-US}} Other users of the Elasticsearch ecosystem, including Logz.io, CrateDB and Aiven, also committed to the need for a fork, leading to a discussion of how to coordinate the open source efforts.{{Cite web|last=Vaughan-Nichols|first=Steven J.|title=AWS, as predicted, is forking Elasticsearch|url=https://www.zdnet.com/article/aws-as-predicted-is-forking-elasticsearch/|access-date=2021-01-28|website=ZDNet|language=en}}{{Cite web|date=2021-01-27|title=CrateDB Doubling Down on Permissive Licensing and the Elasticsearch Lockdown|url=https://crate.io/a/cratedb-doubling-down-on-permissive-licensing-and-the-elasticsearch-lockdown/|access-date=2021-01-28|website=CrateDB|language=en-US}}{{Cite web|date=2021-01-25|title=Momentum Builds to Break Elasticsearch Licensing Deadlock|url=https://www.datanami.com/2021/01/25/momentum-builds-to-break-elasticsearch-licensing-deadlock/|access-date=2021-01-31|website=Datanami}} Due to potential trademark issues with using the name "Elasticsearch", AWS rebranded their fork as OpenSearch in April 2021.{{cite web|last=Anderson |first=Tim |title=You know what? Fork this: AWS renames its take on Elasticsearch to OpenSearch following trademark fight |url=https://www.theregister.com/2021/04/13/aws_renames_elasticsearch_fork_opensearch/ |website=The Register |date=2021-04-13 |accessdate=2021-04-13}}[https://news.slashdot.org/story/21/09/12/0321258/amazon-renames-its-open-source-fork-of-elasticsearch-amazon-opensearch-service TheRegister (12 Sep 2021) Amazon Elasticsearch Service is so flexible it wants to be called by a new name]

In August 2024 the GNU Affero General Public License was added to ElasticSearch version 8.16.0 as an option, making Elasticsearch free and open-source again.{{cite web |url=https://github.com/elastic/elasticsearch/commit/0279c0a909adebd89cc139e05d7f1d170f5fed96 |title=Add AGPLv3 as a supported license · elastic/elasticsearch@0279c0a |website=GitHub |access-date=2025-04-13}}

Features

{{Technical|date=May 2023}}

Elasticsearch can be used to search any kind of document. It provides scalable search, has near real-time search, and supports multitenancy.{{cite web|url=http://www.elasticsearch.org/|title=Official Website|website=Elasticsearch.org|access-date=2014-02-04}} "Elasticsearch is distributed, which means that indices can be divided into shards and each shard can have zero or more replicas. Each node hosts one or more shards and acts as a coordinator to delegate operations to the correct shard(s). Rebalancing and routing are done automatically". Related data is often stored in the same index, which consists of one or more primary shards, and zero or more replica shards. Once an index has been created, the number of primary shards cannot be changed.{{Cite web|url=https://www.datadoghq.com/blog/monitor-elasticsearch-performance-metrics/|title=How to monitor Elasticsearch performance|date=26 September 2016|website=How to monitor Elasticsearch performance}}

Elasticsearch is developed alongside the data collection and log-parsing engine Logstash, the analytics and visualization platform Kibana, and the collection of lightweight data shippers called Beats. The four products are designed for use as an integrated solution, referred to as the "Elastic Stack".{{Cite web|last=Miller|first=Ron|url=https://techcrunch.com/2016/10/26/elastic-finally-brings-order-to-its-product-line-with-elastic-stack/|title=Elastic brings order to its product line with Elastic Stack|website=TechCrunch|date=26 October 2016|language=en-US|access-date=3 April 2024}} (Formerly the "ELK stack", short for "Elasticsearch, Logstash, Kibana".)

Elasticsearch uses Lucene and tries to make all its features available through the JSON and Java API{{Cite web

|title=RedWinterDawn/elasticsearch:15159380

|url=https://github.com/RedWinterDawn/elasticsearch

|website=GitHub

|date=2015-04-05

|doi=10.5281/zenodo.15159380

|access-date=2025-05-09

|last1=Shay Banon

|last2=Simon Willnauer

|last3=David Pilato

|last4=Martijn van Groningen

|last5=Robert Muir

|last6=Clinton Gormley

|last7=Luca Cavanna

|last8=Adrien Grand

|last9=Boaz Leskes

|last10=Ryan Ernst

|last11=Igor Motov

|last12=Britta Weber

|last13=Alexander Reelsen

|last14=Lee Hinman

|last15=Michael McCandless

|last16=Colin Goodheart-Smithe

|last17=Christoph Büscher

|last18=Nik Everett

|last19=Karel Minarik

|last20=Jason Tedor

|last21=Honza Král

|last22=uboness

|last23=Drew Raines

|last24=Tanguy Leroux

|last25=Alex Ksikes

|last26=Zachary Tong

|last27=Nick Knize

|last28=Jun Ohtani

|last29=Brian Murphy

|last30=markharwood

}}

. It supports facetting and percolating (a form of prospective search),{{cite web |url=http://www.elasticsearch.org/guide/reference/api/percolate/ |title=percolate at elasticsearch.org reference |website=Elasticsearch.org |access-date=2014-02-04 |archive-url=https://web.archive.org/web/20131002171546/http://www.elasticsearch.org/guide/reference/api/percolate/ |archive-date=2 October 2013 |url-status=dead |df=dmy-all }}"Percolating" is a term peculiar to Elasticsearch. Percolating is a reverse search: instead of returning all the documents that match a search query, percolating returns all the (stored) search queries that match a document as their output. Nunn, Xavier; [https://blog.serenacapital.com/detecting-data-leaks-in-real-time-with-a-custom-percolator-758bc6dc4fbb "Detecting data leaks in real time with a custom percolator"], Serena Capital blogs, 2019-January-8 which can be useful for notifying if new documents match for registered queries. Another feature, "gateway", handles the long-term persistence of the index;{{cite web|url=http://www.elasticsearch.org/guide/reference/modules/gateway/|title=elasticsearch Guide: Gateway|website=Elasticsearch.org|access-date=19 April 2013}} for example, an index can be recovered from the gateway in the event of a server crash. Elasticsearch supports real-time GET requests, which makes it suitable as a NoSQL datastore,{{cite web|url=http://karussell.wordpress.com/2011/07/13/jetslide-uses-elasticsearch-as-database/|title=Elasticsearch as database|website=Karussell.wordpress.com|date=13 July 2011|access-date=2014-02-04}} but it lacks distributed transactions.{{cite web |date=2010-07-08 |title=No transaction support |url=http://elasticsearch-users.115913.n3.nabble.com/Distributed-transaction-support-td951313.html |archive-url=https://web.archive.org/web/20200323100148/http://elasticsearch-users.115913.n3.nabble.com/Distributed-transaction-support-td951313.html |archive-date=23 March 2020 |access-date=2014-02-04 |website=Elasticsearch-users.115913.n3.nabble.com}}

On 20 May 2019, Elastic made the core security features of the Elastic Stack available free of charge, including TLS for encrypted communications, file and native realm for creating and managing users, and role-based access control for controlling user access to cluster APIs and indexes.{{Cite web|url=https://www.elastic.co/blog/security-for-elasticsearch-is-now-free|title=Security for Elasticsearch is now free|date=2019-05-20|website=Elastic Blog|language=en-us|access-date=2019-06-17}} The corresponding source code is available under the “Elastic License”, a source-available license.{{Cite web|url=https://www.elastic.co/blog/doubling-down-on-open|title=Doubling Down on Open|date=2018-02-27|website=Elastic Blog|language=en-us|access-date=2019-10-24}} In addition, Elasticsearch now offers SIEM{{cite web | title=Introducing Elastic SIEM | website=Elastic Blog | date=2019-06-25 | url=https://www.elastic.co/blog/introducing-elastic-siem | access-date=2020-03-02}} and Machine Learning{{cite web | title=Introducing Machine Learning for the Elastic Stack | website=Elastic Blog | date=2017-05-04 | url=https://www.elastic.co/blog/introducing-machine-learning-for-the-elastic-stack | access-date=2020-03-02}} as part of its offered services.

See also

References

{{Reflist}}