V2.4.6 cloud-azure plugin issue due to hard-coded storage account URL

cptrk · February 26, 2019, 10:59pm

Hi all,

I am aware of the outdated ES and plugin version that I will be discussing here. Due to vendor restrictions this is what I have to work with. I am having an issue with the cloud-azure plugin where there is a hard-coded storage account URL that is not correct for my usage. Regardless of the value for the account config line below, blob.core.windows.net is appended onto the end of it.

elasticsearch 2.4.6
cloud-azure plugin 2.4.6
centos 7.6

The relevant elasticsearch.yml portion (aaaa is for reference later). I have verified I am using the correct account and key.

cloud:
azure:
storage:
my_account:
account: aaaa
key: ****

When attempting a simple snapshot backup using an example from the documentation (--noproxy is required due to the environment)

curl --noproxy "*" -X PUT "localhost:9200/_snapshot/my_backup1" -H 'Content-Type: application/json' -d'
{
"type": "azure"
}
'
output:

{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[my_backup1] can not initialize container elasticsearch-snapshots"}],"type":"repository_verification_exception","reason":"[my_backup1] can not initialize container elasticsearch-snapshots","caused_by":{"type":"storage_exception","reason":"The server encountered an unknown failure: ","caused_by":{"type":"unknown_host_exception","reason":"aaaa.blob.core.windows.net"}}},"status":500}[

The thing is, our storage account is on a whole different domain (blob.core.usgovcloudapi.net). For the life of me, I cannot find any config files where I can change the base URL. I have tried giving the FQDN as the account config line and I end up with the FQDN I supplied prefixed onto blob.core.windows.net.

I even went as far as downloading the entire source code for 2.4.6 and searching for any windows.net URL from there. This is all I can find, one reference in one java file to the core.windows.net domain.

/elasticsearch-2.4/plugins/src/main/java/org/elasticsearch/cloud/azure/management/AzureComputeServiceImpl.java

53: private static final String ENDPOINT = "https://management.core.windows.net/";

Is there any fix out there or a patch I can apply to get things working? I am very interested in working through this problem with anyone who is willing to help. I can supply more information if needed.

Thank you.

dadoonet · February 27, 2019, 9:02am

I believe that if you upgrade you will be able to set azure.client.default.endpoint_suffix: blob.core.usgovcloudapi.net

See https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-azure-usage.html

cptrk · March 11, 2019, 5:17pm

Hi David,
We are unable to upgrade ES due to our vendor that requires v2.4.6. The way it seems given your reply, we are out of luck with our current version as far as this issue is concerned and we cannot change the URL the plugin points to.

dadoonet · March 12, 2019, 1:58am

If you are a Java developer you can try to make the same changes or hardcode the endpoint with your setting may be? And change this line:

github.com

elastic/elasticsearch/blob/v2.4.6/plugins/cloud-azure/src/main/java/org/elasticsearch/cloud/azure/management/AzureComputeServiceImpl.java#L53


import static org.elasticsearch.cloud.azure.management.AzureComputeService.Management.KEYSTORE_TYPE;
import static org.elasticsearch.cloud.azure.management.AzureComputeService.Management.SUBSCRIPTION_ID;


/**
*
*/
public class AzureComputeServiceImpl extends AbstractLifecycleComponent<AzureComputeServiceImpl>
   implements AzureComputeService {


   static final class Azure {
       private static final String ENDPOINT = "https://management.core.windows.net/";
   }


   private final ComputeManagementClient client;
   private final String serviceName;


   @Inject
   public AzureComputeServiceImpl(Settings settings) {
       super(settings);
       String subscriptionId = settings.get(SUBSCRIPTION_ID);

system · April 9, 2019, 1:58am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.