Elastic search snapshot and restore using Azure repository

Elastic Stack Elasticsearch
1

I want to take snapshot and restore using azure repository in Linux, I have configured these below setting in yml, but in the doc it says that this configuration is for previous version,but when i tried the new version configuration, elastic search service is not starting.

with the below setting, service is started, i m getting below error when creating repository.my azure plugin and elastic search version is 2.3.3.

cloud:
    		azure:
        		storage_account: your_azure_storage_account
        		storage_key: your_azure_storage_key

{
  "error": {
    "root_cause": [
      {
        "type": "repository_exception",
        "reason": "[my_backup] failed to create repository"
      }
    ],
    "type": "repository_exception",
    "reason": "[my_backup] failed to create repository",
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "Unknown [repository] type [azure]"
    }
  },
  "status": 500
}
2

Could you try:

cloud:
    azure:
        storage:
            my_account:
                account: your_azure_storage_account
                key: your_azure_storage_key

Source: https://www.elastic.co/guide/en/elasticsearch/plugins/current/cloud-azure-repository.html

1 Like
3

This is working,

cloud:
azure:
storage:
account: your_azure_storage_account
key: your_azure_storage_key

but i'm not able to create a repository. getting this error This might indicate that the store [elasticsearch-snapshots] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node

I was trying to take back up in blob.

4

using ?verify=false parameter, repository verification is skipped, whether this is correct approach?

5

So the settings are corrects but may be you have more than one node and you did not restart all the nodes?

6

I have restarted all the nodes, but still getting this below error

This might indicate that the store is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node

7

Can you see anything in logs?

8

No, i could see this error in log, but it seems to be related to service restarted, when i check the node status, all the nodes are joined.

[2016-06-28 04:20:27,000][WARN ][discovery.zen.ping.unicast] [master-node-1] failed to send ping to [{#zen_unicast_2#}{10.0.1.5}{10.0.1.5:9300}]
RemoteTransportException[[master-node-0][10.0.1.5:9300][internal:discovery/zen/unicast]]; nested: IllegalStateException[received ping request while not started];
Caused by: java.lang.IllegalStateException: received ping request while not started

What is the cause of the store sharing between data node and master nodes? since it is azure blob, it should be working right?

9

What is the cause of the store sharing between data node and master nodes?

That's what we are trying to find here.

since it is azure blob, it should be working right?

Yes.

Can you run:

PUT /_cluster/settings
{
  "transient": {
    "logger.repositories.azure": "TRACE",
    "logger.cloud.azure": "TRACE"
  }
}

And run the create repository command again?
And paste all logs here (formatted please) or on gist.github.com.

If nothing more appears, could you change logging.yml file and add:

   repositories.azure: TRACE
   cloud.azure: TRACE

Then restart your node. And do the same create repository command again?

10

In logs,

[2016-06-28 06:42:33,502][TRACE][cloud.azure.storage      ] [master-node-1] selecting a client for account [null], mode [PRIMARY_ONLY]
[2016-06-28 06:42:33,653][WARN ][repositories             ] [master-node-1] [my_backuplogs] failed to verify repository
RepositoryVerificationException[[my_backuplogs] a file written by master to the store [mybackuplogs] cannot be accessed on the node [{master-node-1}{ai44ymiSTfGg0dgMF9$
        at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.verify(BlobStoreIndexShardRepository.java:247)
        at org.elasticsearch.repositories.VerifyNodeRepositoryAction.doVerify(VerifyNodeRepositoryAction.java:121)
        at org.elasticsearch.repositories.VerifyNodeRepositoryAction.access$200(VerifyNodeRepositoryAction.java:50)
        at org.elasticsearch.repositories.VerifyNodeRepositoryAction$VerifyNodeRepositoryRequestHandler.messageReceived(VerifyNodeRepositoryAction.java:156)
        at org.elasticsearch.repositories.VerifyNodeRepositoryAction$VerifyNodeRepositoryRequestHandler.messageReceived(VerifyNodeRepositoryAction.java:152)
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33)
        at org.elasticsearch.shield.transport.ShieldServerTransportService$ProfileSecuredRequestHandler.messageReceived(ShieldServerTransportService.java:180)
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
        at org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:245)
        at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:114)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
11

I updated your answer. Please format your messages with </> button to make them easier to read.

May be you could paste all the logs from all nodes to gist.github.com?

Check that every node has write access to the azure repo.

12

Thank you so much for the help..

It works, issue is blob does not have write access.Thanks