# Index rollover using curator

**URL:** <https://discuss.elastic.co/t/index-rollover-using-curator/98392>\
**Category:** Elasticsearch\
**Created:** [August 25, 2017, 3:25pm UTC](https://discuss.elastic.co/t/index-rollover-using-curator/98392 "2017-08-25T15:25:23Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ihjaz](https://avatars.discourse-cdn.com/v4/letter/i/8baadc/32.png) [@Ihjaz](https://discuss.elastic.co/u/Ihjaz)\
**Post date:** [August 25, 2017, 3:25pm UTC](https://discuss.elastic.co/t/index-rollover-using-curator/98392/1 "2017-08-25T15:25:23Z")

</div>

Hi,

I'm trying to get the indices rolled over once it reaches a given number of documents. I have the curator action as shown below:

```
  2:
    action: rollover
    description: >-
      Rollover the index associated with index 'name', which should be in the
      form of prefix-000001 (or similar), or prefix-YYYY.MM.DD-1.
    options:
      disable_action: False
      name: filebeat_logs
      conditions:
        max_docs: 500

```

I have added an alias named filebeat\_logs using the curator action.

```
1:
    action: alias
    description: >-
      Alias indices from last week, with a prefix of logstash- to 'last_week',
      remove indices from the previous week.
    options:
      name: filebeat_logs
      warn_if_no_indices: False
      disable_action: False
    add:
      filters:
      - filtertype: pattern
        kind: prefix
        value: filebeat-

```

The index is created by the logstash as shown below :

```
input {
  beats {
    host => "10.10.132.40"
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => "10.10.132.44:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}-1"
    document_type => "%{[@metadata][type]}"
  }
} 

```

When I run the curator, it just creates a new empty index and the logs still gets written to the older index.

```
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open filebeat-2017.08.25-000002 UyCwyENbT1ah96nT79MyqQ 5 1 0 0 955b 955b
yellow open filebeat-2017.08.25-1 XNB0rSxJRIizISIGqbMOSA 5 1 906 0 775.8kb 775.8kb

```

I read that I need to right to an alias and then it will start writing to the new file.  
But can someone tell me how I can define and write to an alias from the elasticsearch output plugin so that the logs get written to new files every time I run the curator rollover action.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [August 25, 2017, 3:49pm UTC](https://discuss.elastic.co/t/index-rollover-using-curator/98392/2 "2017-08-25T15:49:58Z")

</div>

> [@Ihjaz](#):
>
> I read that I need to right to an alias and then it will start writing to the new file.

That is correct.

> [@Ihjaz](#):
>
> But can someone tell me how I can define and write to an alias from the elasticsearch output plugin so that the logs get written to new files every time I run the curator rollover action.

This is currently impossible. You need to manually create the initial index + alias to do rollover. The upside is that once this is done, it never needs to be done again—it's a one-time only thing (unless you accidentally delete the alias). The example below is taken from [the official Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/indices-rollover-index.html#indices-rollover-index):

```auto
PUT /filebeat-000001 
{
  "aliases": {
    "filebeat_logs": {}
  }
}

```

This will create index `filebeat-000001` and alias `filebeat_logs` will point to it.

Your logstash output block would then look like:

```auto
output {
  elasticsearch {
    hosts => "10.10.132.44:9200"
    manage_template => false
    index => "filebeat_logs"
  }
} 

```

That's it. You point the `index` directive to the _alias._

**Extra credit:**  
Since you've disabled template management, you should manually maintain an index template that has appropriate mappings for indices following the `filebeat-*` pattern.

I highly recommend getting away from using `document_type => "%{[@metadata][type]}"`, as [mapping types are going away in Elasticsearch 6.0](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html), meaning that each index should have one and only one document type. If you were planning on having multiple types per index, now is a good time to adjust that to multiple indices with one type per index, before you get a bunch of indices which won't be upgradable.

---

<div class="post-metadata">

**Author:** ![Ihjaz](https://avatars.discourse-cdn.com/v4/letter/i/8baadc/32.png) [@Ihjaz](https://discuss.elastic.co/u/Ihjaz)\
**Post date:** [August 30, 2017, 10:37am UTC](https://discuss.elastic.co/t/index-rollover-using-curator/98392/3 "2017-08-30T10:37:52Z")

</div>

Thanks @theuntergeek.

This is exactly what I wanted.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 27, 2017, 10:38am UTC](https://discuss.elastic.co/t/index-rollover-using-curator/98392/4 "2017-09-27T10:38:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.