Keep it free and open

OpenSearch & OpenSearch Dashboards, who? 🤔¶

On 12 April 2021, Amazon announces OpenSearch project, a community-driven, open source fork of Elasticsearch and Kibana.

This project includes OpenSearch, derived from Elasticsearch 7.10.2, and OpenSearch Dashboards, derived from Kibana 7.10.2. Additionally, the OpenSearch project is the new home for Amazon's distribution of Elasticsearch, Open Distro for Elasticsearch, which includes features such as enterprise security, alerting, machine learning, SQL, index state management and more.

All of the software in the OpenSearch project is released under the Apache License, Version 2.0 (ALv2).

Why do we need OpenSearch? Isn't it just a rebranding-fork? 🙄¶

Let's make it short, shall we? 🤏¶

On January 2021, Elastic announced that they will change their software licensing strategy and will not release the new versions of Elasticsearch and Kibana under the Apache License, Version 2.0 (ALv2). Instead, the new versions of the software will be offered under the Elastic License, which limits how it can be used, or the Server Side Public License (SSPL), which has requirements that make it unacceptable to many in the open source community. This means that Elasticsearch and Kibana will no longer be open source software.

Switching to SSPL, Elastic claims that their products continue to remain “open” under the new license, but the new license actually has taken away user rights.Licenses such as SSPL allow a user to view the source code but do not allow other highly important rights protected by the Open Source Definition, such as the right to make use of the program for any field of endeavor.

In a post it unironically titled “Doubling Down on Open,” Elastic says that it now can “restrict cloud service providers from offering our software as a service” in violation of Open Source Definition art.6. Elastic didn’t double down, it threw its cards in.

How did the community react? 👀¶

The situation at time of writing ⏰¶

Amazon is not alone behind OpenSeach Project, organizations such as Red Hat, SAP, Capital One, and Logz.io have joined in support for this huge project.

On 13 May 2021, Amazon announces the first General Available (GA) release: OpenSearch 1.0.0 Beta 1!

OpenSearch and OpenSearch Dashboards based on respectively on ElasticSeatch 7.10.2 and Kibana 7.10.2
Tarball archives ready to install
Docker images ready to be pulled
Docker-compose example provided (The one we are going to use)

Let's start a OpenSearch instance!

0. Get Docker and Compose working on your machine 😬¶

Ahahahaha, nice joke!

How did you get here while attending TAP 2021+ course without a running Docker?

Don't cheat.

1. Get the Docker-Compose file 📦¶

We are going to creare a tmpOS folder and download the file inside there.

``` mkdir "tmpOS" && \ cd ./tmpOS && \ curl -O "https://www.opensearch.org/samples/docker-compose.yml

Requirements:

curl (apt install curl | dnf install curl | apk add curl)

In [5]:

%%bash
mkdir "tmpOS" && \
cd "./tmpOS" && \
curl -O "https://www.opensearch.org/samples/docker-compose.yml" &> /dev/null

2. So easy, let's compose up! ⚠️¶

Well... not that easy.

On WSL2 machines or Linux machines, OpenSearch nodes will refuse to run exiting with code 1.

Why?

~Elasticsearch~ OpenSearch uses a mmapfs directory by default to store its indices. The default operating system limits on mmap counts is likely to be too low, which may result in out of memory exceptions.

To increase the limit we need to run the following command as root:

sysctl -w vm.max_map_count=262144

In [23]:

import getpass
import os
password = getpass.getpass()
command = "sudo -S sysctl -w vm.max_map_count=262144"
os.system('echo %s | %s' % (password, command))

········

Out[23]:

2.1. Let's compose up! ☕️¶

Now, we are finally ready to start up our example. Just run:

docker-compose up

Quoting OpenSearch's blog: Have a nice coffee while everything is downloading and starting up

In [ ]:

%%bash
docker-compose up

3. "Hacking the system..." 🧑🏻‍💻¶

We can now log in at http://localhost:5601/
Using as Username: admin (super secure!)
Using as Password: admin (super secure!)

OpenSearch Beta 1, what changes?¶

Nothing, for now.

All of the concepts seen in class are still applicable.

The documentation warns that the project is still in development and nothing is final yet. Breaking-changes may be introduced as OpenSearch becomes more established.

The new licenses introduced by Elastic do not favor the open-source world in any way, we cannot predict future changes but one thing is certain: very soon the open-source community will migrate its interest to OpenSearch.

As said before: ElasticSearch is not a open source software anymore

Thank you for your attention¶