# Use case for elasticsearch-curator using AWS S3 repository

**URL:** <https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635>\
**Category:** Elasticsearch\
**Created:** [June 15, 2017, 10:58pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635 "2017-06-15T22:58:13Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jaff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaff/32/18382_2.png) [@Jaff](https://discuss.elastic.co/u/Jaff)\
**Post date:** [June 15, 2017, 10:58pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/1 "2017-06-15T22:58:13Z")

</div>

After upgrade to ES 5.4.1, I was told there is new curator version, and we would like to take advantage of added capability. Looking into configuration examples, what would be the best example of how to set this up?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 15, 2017, 11:32pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/2 "2017-06-15T23:32:57Z")

</div>

Hi Mark. What version of Elasticsearch were you using before you upgraded? And which version of Curator?

---

<div class="post-metadata">

**Author:** ![Jaff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaff/32/18382_2.png) [@Jaff](https://discuss.elastic.co/u/Jaff)\
**Post date:** [June 20, 2017, 6:06pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/3 "2017-06-20T18:06:42Z")

</div>

Previous version of ES was 2.7.3, and used curator 3.3.0.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 20, 2017, 10:18pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/4 "2017-06-20T22:18:07Z")

</div>

Did you mean 1.7.3? There's never been an Elasticsearch 2.7.x. The 2.x branch ended with 2.4.

Curator 5.1.1 is the current release. The documentation for it is [online](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.1/index.html), where you ostensibly found the configuration examples. I'm confused, though. If you have read and understand the configuration examples, what do you need help with? Your request, "what would be the best example of how to set this up?" is very open-ended.

---

<div class="post-metadata">

**Author:** ![Jaff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaff/32/18382_2.png) [@Jaff](https://discuss.elastic.co/u/Jaff)\
**Post date:** [June 20, 2017, 11:41pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/5 "2017-06-20T23:41:12Z")

</div>

Sorry, I did not look at the right file; it's 2.3.3 version.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 20, 2017, 11:59pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/6 "2017-06-20T23:59:55Z")

</div>

Thanks for the version confirmation. My previous question remains:

> [@theuntergeek](#):
>
> I'm confused, though. If you have read and understand the configuration examples, what do you need help with? Your request, "what would be the best example of how to set this up?" is very open-ended.

Curator v4.3.1 (latest version in the 4.x branch) will work with Elasticsearch 2.3.

---

<div class="post-metadata">

**Author:** ![Jaff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaff/32/18382_2.png) [@Jaff](https://discuss.elastic.co/u/Jaff)\
**Post date:** [June 21, 2017, 12:04am UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/7 "2017-06-21T00:04:27Z")

</div>

I am talking about configuring for Curator 5.1.1 in ES 5.4.1, where we are using S3 for repository. How would we make the connection? Or is the assumption that storage is local to the curator host?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 21, 2017, 12:57am UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/8 "2017-06-21T00:57:03Z")

</div>

Thank you for clarifying.

First, you must install the [S3 Repository Plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/5.4/repository-s3.html).

Then you must [configure it](https://www.elastic.co/guide/en/elasticsearch/plugins/5.4/repository-s3-usage.html).

And lastly, [create a repository](https://www.elastic.co/guide/en/elasticsearch/plugins/5.4/repository-s3-repository.html).

Once a repository has been created, Curator only needs to reference it by name:

```auto
---
actions:
  1:
    action: snapshot
    description: >-
      Snapshot indices as filtered into named repository, with 
      default curator-%Y%m%d%H%M%S snapshot name
    options:
      repository: REPO_NAME_GOES_HERE
      name:
      wait_for_completion: True
      max_wait: 3600
      wait_interval: 10
    filters:
    - filtertype: ...

```

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 21, 2017, 1:48am UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/9 "2017-06-21T01:48:34Z")

</div>

I should add that the S3 repository plugin and configuration steps must be done on _all_ master and data nodes in your cluster. The repository creation step is API-based, and therefore only needs to be done once for it to apply cluster-wide.

---

<div class="post-metadata">

**Author:** ![Jaff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaff/32/18382_2.png) [@Jaff](https://discuss.elastic.co/u/Jaff)\
**Post date:** [June 21, 2017, 8:50pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/10 "2017-06-21T20:50:09Z")

</div>

I see your pointers, that's what we're doing. I should mention that all cluster setup is done via automation, with Chef. I had repo created, but when I referenced it, I get exception. It turns out I had a typo in parameters, and it is now sorted out. Sorry for the lost bandwidth.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 19, 2017, 8:50pm UTC](https://discuss.elastic.co/t/use-case-for-elasticsearch-curator-using-aws-s3-repository/89635/11 "2017-07-19T20:50:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
