Azure Elasticsearch Template Failing

Elastic Stack Elasticsearch
1

Hi,

I recently deployed the elasticsearch template into our Azure environment. Originally, I was experiencing issues where I couldn't access my ES nodes, but Kibana appeared to be working fine. At this point everything was deployed onto Linux machines. I wanted to remote into these machines, so I decided to delete everything and try them on windows OS.

So I removed all instances of anything ES related and redeployed. Now I get this error:
"Unable to process template language expressions for resource '/subscriptions/92841702-a625-4dea-afde-ec7bc9d6ee65/resourceGroups/qmtdev/providers/Microsoft.Compute/virtualMachines/elasticsearch-kibana-vm/extensions/installkibana' at line '166' and column '10'. 'Template language expression property 'kibana' doesn't exist, available properties are 'marvel, master, client, data'."

Any ideas would be great.

Thanks everyone,

Matthew

2

@MatthewHartz, we've been having a discussion about this via email but I'll put the salient pieces here as it may help others.

Based on your description, it sounds like you might be using the Azure Quickstart template which is a community contributed template that supports deployment to Windows machines. This is a different template from Elastic's Azure ARM template offering, which you can find on the Marketplace to deploy using the Azure command line interface (CLI), or alternatively, find within the Azure portal by searching for "Elasticsearch and Kibana", and look for the official one published by Elastic. Elastic's template currently deploys only to Ubuntu Linux machines and can be deployed into an existing Virtual Network; I wrote a blog post recently highlighting some of the main features of the template.

With deploying to an existing virtual network, the address space of the network and subnet are declared when created; you can add an additional address range to a Virtual network as well as add additional subnets (network range permitting), but you cannot modify existing ranges.

With that in mind, when deploying to an existing virtual network

  1. the network and subnet need to exist within the same subscription and location as the resource group into which other template resources will be deployed

  2. the network and subnet will need to have sufficient private IP addresses available for all resources deployed as part of the template; for the default configuration (3 dedicated master nodes, 3 data nodes, 1 kibana node and 1 internal load balancer) you will need 8 available private IP addresses. The IP addresses of the master and data nodes, as well as kibana VM are all dynamically assigned, but you will need to provide a static IP address for the internal load balancer; if deploying from the portal UI, when you select a subnet, the next available private IP address in the subnet will be passed automatically as the load balancer IP, but if using the deployment template directly, you will need to provide an available IP on the subnet to the template as the value to property name vNetLoadBalancerIp. This is needed as Kibana accesses the cluster via the internal load balancer so the IP address of the load balancer is written to the kibana configuration file.

The template installs with Shield, part of our commercial X-Pack offering that handles authentication and authorization. As part of template deployment, you supply credentials for three roles; an admin role, a kibana role and a readonly role. Each of these has different levels of access to the cluster, from most to least access, respectively, so you can use whichever one makes the most sense for what you need to do. The documentation has more details around adding additional users and roles.

One thing to consider that is also highlighted in the blog post, is that the deployment does not set up Transport Layer Security; if the cluster will be accessible externally, for example, via a public IP for Kibana or an external load balancer, you should consider setting this up to secure communication.

Using Application Gateway is probably the least hassle way of getting TLS setup in Azure