Use case for elasticsearch-curator using AWS S3 repository

Elastic Stack Elasticsearch
1

After upgrade to ES 5.4.1, I was told there is new curator version, and we would like to take advantage of added capability. Looking into configuration examples, what would be the best example of how to set this up?

2

Hi Mark. What version of Elasticsearch were you using before you upgraded? And which version of Curator?

3

Previous version of ES was 2.7.3, and used curator 3.3.0.

4

Did you mean 1.7.3? There's never been an Elasticsearch 2.7.x. The 2.x branch ended with 2.4.

Curator 5.1.1 is the current release. The documentation for it is online, where you ostensibly found the configuration examples. I'm confused, though. If you have read and understand the configuration examples, what do you need help with? Your request, "what would be the best example of how to set this up?" is very open-ended.

5

Sorry, I did not look at the right file; it's 2.3.3 version.

6

Thanks for the version confirmation. My previous question remains:

Curator v4.3.1 (latest version in the 4.x branch) will work with Elasticsearch 2.3.

7

I am talking about configuring for Curator 5.1.1 in ES 5.4.1, where we are using S3 for repository. How would we make the connection? Or is the assumption that storage is local to the curator host?

8

Thank you for clarifying.

First, you must install the S3 Repository Plugin.

Then you must configure it.

And lastly, create a repository.

Once a repository has been created, Curator only needs to reference it by name:

---
actions:
  1:
    action: snapshot
    description: >-
      Snapshot indices as filtered into named repository, with 
      default curator-%Y%m%d%H%M%S snapshot name
    options:
      repository: REPO_NAME_GOES_HERE
      name:
      wait_for_completion: True
      max_wait: 3600
      wait_interval: 10
    filters:
    - filtertype: ...
9

I should add that the S3 repository plugin and configuration steps must be done on all master and data nodes in your cluster. The repository creation step is API-based, and therefore only needs to be done once for it to apply cluster-wide.

10

I see your pointers, that's what we're doing. I should mention that all cluster setup is done via automation, with Chef. I had repo created, but when I referenced it, I get exception. It turns out I had a typo in parameters, and it is now sorted out. Sorry for the lost bandwidth.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.