Adding non-elastic plugins to Elasticsearch & Kibana

[Wes_F]

I was trying to figure out the best way to add a plugin that is not in the list of pre-loaded plugins. And I saw that in Elastic Cloud you can upload plugins but not in Elastic Cloud Enterprise.

Is there a way to add plugins that I am just not seeing: https://www.elastic.co/guide/en/elasticsearch/plugins/5.5/api.html
For example, we are looking for the SQL and the Taste plugins for Elasticsearch.

[Yuri (Yuri Tceretian)]

Hello!
ECE does support user plugins. Although there is no UI for that, you can use API (see API Reference - Create Elasticsearch Cluster and ElasticsearchConfiguration that is part of the payload for the "create cluster" API endpoint).

For example, to add SQL plugin to an existing cluster (v 5.4.1) you need to

• get cluster Id. Go to cluster "Overview" page. Url (/region/ece-region/cluster/{{es_cluster_id}}) will contain cluster Id.
• get the current plan. API request:

curl -u root:{root_password} -X GET http://{{coordinator_host}}/api/v1/clusters/elasticsearch/{{es_cluster_id}}/plan

The request should return a JSON object that we call 'plan':

{
  "zone_count": 1,
  "cluster_topology": [{
    "memory_per_node": 1024,
    "node_count_per_zone": 1,
    "node_configuration": "default"
  }],
  "elasticsearch": {
    "version": "5.4.1"
  }
}

• Modify the plan. Add an array field elasticsearch.user_plugins with a JSON object as element

{
    "name" : "sql-plugin",
    "url": "https://github.com/NLPchina/elasticsearch-sql/releases/download/5.4.1.0/elasticsearch-sql-5.4.1.0.zip",
    "elasticsearch_version": "5.4.1"
}

So the plan above should look like:

{
    "zone_count": 1,
    "cluster_topology": [
        {
            "memory_per_node": 1024,
            "node_count_per_zone": 1,
            "node_configuration": "default"
        }
    ],
    "elasticsearch": {
        "version": "5.4.1",
        "user_plugins": [
            {
                "name": "sql-plugin",
                "url": "https://github.com/NLPchina/elasticsearch-sql/releases/download/5.4.1.0/elasticsearch-sql-5.4.1.0.zip",
                "elasticsearch_version": "5.4.1"
            }
        ]
    }
}

• Submit the plan.

curl -u root:{root_password} -X POST http://{{coordinator_host}}/api/v1/clusters/elasticsearch/{{es_cluster_id}}/plan \
  -H 'content-type: application/json' \
  -d ' {
    "zone_count": 1,
    "cluster_topology": [
        {
            "memory_per_node": 1024,
            "node_count_per_zone": 1,
            "node_configuration": "default"
        }
    ],
    "elasticsearch": {
        "version": "5.4.1",
        "user_plugins": [
            {
                "name": "sql-plugin",
                "url": "https://github.com/NLPchina/elasticsearch-sql/releases/download/5.4.1.0/elasticsearch-sql-5.4.1.0.zip",
                "elasticsearch_version": "5.4.1"
            }
        ]
    }
}'

The server should return cluster's ID in the response. It means that the plan was accepted. You can see progress on the cluster "Overview" page.

To create a cluster with the plugin you need to submit the same plan to /api/v1/clusters/elasticsearch

[Yuri (Yuri Tceretian)]

Actually, there is much simpler way to do the same via UI. Go to "Manage" tab on the cluster details page and click on "advanced cluster configuration" link. You will see field "Plan" that contains pretty big JSON object because it shows all default values. Find user_plugins and modify plan the same way as above and then just click "Save" button.

[Wes_F]

Thank you, That worked magically!