The version of elastic search available for deployment on Azure as a marketplace template is limited to the latest version of 2.3.1 Why is this so far back in the evolution?
Are you using https://azuremarketplace.microsoft.com/en-us/marketplace/apps/elastic.elasticsearch?tab=Overview?
1 Like
No, I am using this: https://github.com/Azure/azure-quickstart-templates
I am running a benchmarking rally on a cluster deployed on Azure. I do not wish to have x-pack installed as this involves redeploying the solution after expiry and its simply unnecessary during the benchmarking phase.
Also again, Why is logstash not shipped along with this?
Well we don't manage that one you linked to, so I cannot comment.
Which one?
I initially deployed a cluster straight from Azure portal using the ARM. I think this is the same one that you pointed me to in your first comment. I do not want x-pack installed on this cluster. Perhaps, I can disable this in the options the template provides. I am not sure at the moment, please correct me if I am wrong.
My pursuit is towards being able to deploy the ES cluster on Azure using a deployment script. In this regard, I started off by deploying one via the ARM. Then exported the automation script and saved it as a JSON. I then attempted to deploy this cluster to another resource group on Azure, however the values such as
"extensions_installelasticsearch_commandToExecute": {
"defaultValue": null,
"type": "SecureString"
}
appear to be null and the Azure CLI throws an exception of a missing value.
Any pointers on how I could achieve this?
I would not recommend exporting the automation script from the Azure Portal; not every resource in Azure and thus in an ARM template is exportable by the Azure Portal, and additionally, the script may write null values for sensitive parameters as you have found.
If you are looking at using the ARM template at https://github.com/elastic/azure-marketplace, I would recommend to
-
have a thorough read through of the README on the repo, to understand all of the available parameters and recommendations
-
have a read of the following blog posts
-
Take a look at the examples in https://github.com/elastic/azure-marketplace-examples for scripting with the template for different cluster topologies. The examples are with the Azure PowerShell SDK, but could be easily be written in your favourite scripting language and use the Azure CLI (1 or 2).
1 Like
Thanks for the descriptive reply. I checked the links and went through the posts. I could deploy the ES version 5.5.1 on Azure using an automation script via the Azure CLI. Although I could do this successfully for the cluster deployed in the location West US, I could not do the same for the one in West Europe. I get the following error.
Deployment failed. {
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "The resource operation completed with terminal provisioning state 'Failed'.",
"details": [
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.",
"details": [
{
"code": "BadRequest",
"message": "{\r\n "error": {\r\n "code": "NoRegisteredProviderFound",\r\n "message": "No registered resource provider found for location 'westeurope' and API version '2017-09-01' for type 'publicIPAddresses'. The supported api-versions are '2014-12-01-preview, 2015-05-01-preview, 2015-06-15, 2016-03-30, 2016-06-01, 2016-07-01, 2016-08-01, 2016-09-01, 2016-10-01, 2016-11-01, 2016-12-01, 2017-03-01, 2017-04-01, 2017-06-01, 2017-08-01, 2017-09-01'. The supported locations are 'westus, eastus, northeurope, westeurope, eastasia, southeastasia, northcentralus, southcentralus, centralus, eastus2, japaneast, japanwest, brazilsouth, australiaeast, australiasoutheast, centralindia, southindia, westindia, canadacentral, canadaeast, westcentralus, westus2, ukwest, uksouth, koreacentral, koreasouth'."\r\n }\r\n}"
}
]
}
]
}
}
Apparently there is no provider for public IP addresses on the west Europe location for the given version. I could not locate where this version is set as this is not present in the mainTemplate.json neither in external-lb-resources.json. The version in external-lb-resources.json is 2015-06-15 as suggested in the error message. Therefore I am clueless as to what went amiss.
Any pointers?
Thank you for the detailed error message @sanketshinde, I've opened an issue on the Azure Marketplace github repository for this.
I've not seen this issue before and have deployed the template frequently to WestEurope in the past, so I think this may be a fairly recent change (possibly a bug?) in the Azure Infrastructure. Looking through the template though, the only API versions there are for public IP addresses are:
- 2015-06-15
- 2016-03-30
Could you try to deploy again and see if you get the same issue?
1 Like
@forloop isn't this already configured here https://github.com/elastic/azure-marketplace/blob/master/src/loadbalancers/external-lb-resources.json ? Or how do you suggest I change the api version manually?
Yes, the template specifies API versions to use already.
On further investigation, the issue is related to a change in the Azure infrastructure that affects the way that resources are referenced/resolved within outputs sections. I've opened a PR to address the issue.
1 Like
I just redeployed this and it worked. I could deploy to the West Europe location.
@forloop Thank you for the assistance. This was of great help. For all other folks that may wonder in future what happened and how I managed to resolve this, surprisingly I did not change anything in the script. Perhaps this was something that was wrong with the Azure cloud service on that particular day, for the entire day. Perhaps it was a bug that was quickly addressed after @forloop raised a issue. I am not sure what exactly happened that solved the problem. However, it works for me now. I deployed a cluster using the automation script linked in the previous replies via the Azure CLI. I am using a MacBook Pro.
Hope the information helps someone who might stumble on to this.
Good to hear it's now working @sanketshinde 
To clarify, I received confirmation from Microsoft that this was related to new API versions rolled out in some regions but not all, so the issue is likely to have been intermittent. Still, the PR addresses this issue from happening in the future.
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.