This year a fork of Elasticsearch was created called OpenSearch. The reasons behind it are varied, and I'm not going to get into that side of things other than to say there are two sides to every story, and it didn't end well.

Our Skpr hosting platform offers a managed AWS Elasticsearch service. As a result of the split, AWS has deprecated Elasticsearch and is migrating all of its managed services to OpenSearch.

This left us in a bit of a difficult situation.

In this blog post, we explain our rationale for choosing to fork Elasticsearch Connector and create a new Search API OpenSearch module.

by
kim.pepper
/ 8 December 2021

At PreviousNext, a number of our projects are currently using the Elasticsearch Connector module. This module is based on the Elasticsearch PHP library, as well as a PHP wrapper library. The module has been around since 2013, supports Drupal 7, 8 and 9 and has almost 5000 reported sites using it.

Unfortunately, the Elasticsearch client library that Elasticsearch Connector uses, will only connect to OpenSearch at version 7.10.2 and below. This means there is a limited amount of support for this library on OpenSearch going forward, and users will need to pin their versions to avoid incompatibilities. This also means users will not be able to take advantage of new features, bug fixes and security updates.

To avoid this issue, the OpenSearch project forked PHP Elasticsearch client library and created a new  opensearch-php  client library.

Trying to support two different PHP client libraries would require some major rewriting of Elasticsearch Connector module and potentially another bridge library. It seemed like it would be more trouble than it was worth.

There was only one real option left: we would need to fork Elasticsearch Connector and create a new module: Search API OpenSearch.

After posting an issue on the Elasticsearch issues list declaring our intentions to fork the module with the maintainers, we set about the task of rewriting the structure of the module. Key goals were:

• Clear separation of responsibilities for code
• Use dependency injection wherever possible
• Avoid static methods
• Ensure a high-level of unit and kernel test code coverage
• Start with the basic features

Elasticsearch Connector provides two config entities: Cluster and Index, for storing configuration. It then wraps these to support Search API. In our module, we wanted to simplify this and leverage the existing Search API Server and Index config entities directly, and tie ourselves more cleanly into the Search API ecosystem.

We also recognised there are many different hosting providers that have their own way of authenticating connections, so we created a new Connector plugin type with basic auth supported out of the box, to allow 3rd-party modules to easily add their own.

Our initial goal, is to provide basic indexing and search functionality. So far the module supports:

• Indexing, including mapping fields
• Searching
• Views
• Search Pages
• Facets
• More Like This

We hope to make the module as extendable as possible via services, events and plugins so that you can shape it to suit your specific needs.

Please try it out, and let us know if you find any issues by posting to the issue queue!

Tagged

Elasticsearch, OpenSearch