Azure Market Place - Elastic Stack for V6.3?

[djl2]

Do you plan to update the Elastic Stack - (Elasticsearch, Kibana, X-Pack) for v6.3 soon? The current template only supports up to v6.2.4.

elastic/azure-marketplace

Also, I tried manually updating my nodes to v6.3.1 using apt-get, but got this message--

Reading state information... Done
elasticsearch is already the newest version (6.2.4).
    0 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.

Will the v6.3.1 packages be added soon?

Thanks.

[forloop (Russ Cam)]

@djl2 Azure Marketplace and the ARM template support up to 6.3.1, with 6.3.1 being the default version:

elastic/azure-marketplace/blob/master/src/mainTemplate.json#L22-L49

Here's what the portal looks like:

The release notes for 6.3.1 can be found in the GitHub repository.

[djl2]

Oddly enough, when I click on my old link on my Azure Dashboard to the Elasticsearch Market Place cluster, it still only shows v6.2.4 as the latest version.

But if search for the ES cluster and bring up the Market Place ES Cluster fresh, I see v6.3.1. The links look identical too.

Anyway, thanks for the reply.

A couple more questions--

I originally forked the v6.2.4 templates and modified them quite a bit in order to deploy as single Nodes to Availability Zones and Scale Sets.

Without looking at all the v6.3.1 ARM templates changes, should I be able to simply add v6.3.1 to the mainTemplate.json and expect the newer versions to deploy? Did things change a lot in the templates in terms of how v6.3.1 is deployed?

Also, what's the easiest way to upgrade an existing v6.2.4 cluster? I tried using apt-get but it said that the latest packages were v6.2.4.

TIA

[forloop (Russ Cam)]

@djl2

I originally forked the v6.2.4 templates and modified them quite a bit in order to deploy as single Nodes to Availability Zones and Scale Sets.

We have an open issue for VM Scale sets - if you'd like to contribute back a PR for this feature, that would be very welcome!

@djl2

Without looking at all the v6.3.1 ARM templates changes, should I be able to simply add v6.3.1 to the mainTemplate.json and expect the newer versions to deploy? Did things change a lot in the templates in terms of how v6.3.1 is deployed?

There are a few changes related to the opening of X-Pack in 6.3.0; the distribution downloaded and installed for 6.3.0+ includes X-Pack features in the package, so X-Pack as a separate plugin to install does not exist.

There are also several other changes related to larger disks, shard allocation awareness, SSL/TLS for HTTP and Transport layers of Elasticsearch that may make it beneficial to apply the AZ and VMSS changes on top of.

@djl2

Also, what's the easiest way to upgrade an existing v6.2.4 cluster? I tried using apt-get but it said that the latest packages were v6.2.4.

The template uses the Debian package so would advise using this for the upgrade. You can perform a rolling upgrade from 6.2.4 to 6.3.2 (latest). A few things to note:

• the template uses monit to monitor the Elasticsearch process, so should be interacted with using monit e.g.

  # to stop
  sudo monit stop elasticsearch
  
  # to start
  sudo monit start elasticsearch
  

• It is highly recommended that you backup your data before performing the upgrade. You may want to use the azure-repository plugin for this if you haven't installed already.

• The Debian package should leave your elasticsearch.yml configuration intact, but I would recommend making a copy anyway, in case.