# Cant roll over index via index template?

**URL:** <https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264>\
**Category:** Elasticsearch\
**Created:** [January 15, 2019, 8:37am UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264 "2019-01-15T08:37:29Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![bmatoki](https://avatars.discourse-cdn.com/v4/letter/b/7ea924/32.png) [@bmatoki](https://discuss.elastic.co/u/bmatoki)\
**Post date:** [January 15, 2019, 8:37am UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/1 "2019-01-15T08:37:29Z")

</div>

Hey all.  
im trying to set roll over in my index template.  
my logstash using index template  
this is my index template

> ```
> > 
> > {
> > "template" : "logstash-*",
> > "version" : 60001,
> > "settings" : {
> > "index.refresh_interval" : "5s"
> > },
> > "mappings" : {
> > "_default_" : {
> > "dynamic_templates" : [ {
> > "message_field" : {
> > "path_match" : "message",
> > "match_mapping_type" : "string",
> > "mapping" : {
> > "type" : "text",
> > "norms" : false
> > }
> > }
> > }, {
> > "string_fields" : {
> > "match" : "*",
> > "match_mapping_type" : "string",
> > "mapping" : {
> > "chain_array": {
> > "type": "text"
> > },
> > "client_time": {
> > "type": "date"
> > },
> > "destination_ip": {
> > "type": "ip"
> > },
> > "destination_path": {
> > "type": "text"
> > },
> > "destination_port": {
> > "type": "integer"
> > },
> > "direction": {
> > "type": "keyword"
> > },
> > "full_server_time": {
> > "type": "date"
> > },
> > "reporting_computer": {
> > "type": "text"
> > },
> > "dll_name": {
> > "type": "text"
> > },
> > "dll_path": {
> > "type": "text"
> > },
> > "mog_counter": {
> > "type": "integer"
> > },
> > "os": {
> > "type": "keyword"
> > },
> > "process_id": {
> > "type": "integer"
> > },
> > "process_name": {
> > "type": "text"
> > },
> > "process_path": {
> > "type": "text"
> > },
> > "protocol": {
> > "type": "keyword"
> > },
> > "reason": {
> > "type": "keyword"
> > },
> > "sequance_number": {
> > "type": "integer"
> > },
> > "source_ip": {
> > "type": "ip"
> > },
> > "source_port": {
> > "type": "integer"
> > },
> > "scramble_state": {
> > "type": "keyword"
> > },
> > "status": {
> > "type": "keyword"
> > },
> > "sub_sequance_number": {
> > "type": "integer"
> > },
> > "user_name": {
> > "type": "text"
> > },
> > "cast_type": {
> > "type": "keyword"
> > },
> > "counter" : {
> > "type": "integer"
> > }
> > }
> > }
> > } ],
> > "properties" : {
> > "@timestamp": { "type": "date"},
> > "@version": { "type": "keyword"},
> > "geoip" : {
> > "dynamic": true,
> > "properties" : {
> > "ip": { "type": "ip" },
> > "location" : { "type" : "geo_point" },
> > "latitude" : { "type" : "half_float" },
> > "longitude" : { "type" : "half_float" }
> > }
> > }
> > }
> > }
> > }
> > }
> 
> ```

if i add conditions the logstash cant install the template.  
there is any option to set rollover via index template instead of put to aliases?

---

<div class="post-metadata">

**Author:** ![bmatoki](https://avatars.discourse-cdn.com/v4/letter/b/7ea924/32.png) [@bmatoki](https://discuss.elastic.co/u/bmatoki)\
**Post date:** [January 16, 2019, 2:25pm UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/2 "2019-01-16T14:25:28Z")

</div>

bump.  
added curator for delete old indices and delete by disk space - work.

remain : rollover index every 5gb or 1day  
my logstash config

> input {  
> tcp {  
> port =\> 5556  
> }  
> udp {  
> port =\> 5566  
> }  
> }
> 
> filter {  
> csv {  
> separator =\> ","  
> columns =\> [  
> "os","reporting\_computer","client\_time" ,"full\_server\_time" ,"process\_id" ,"process\_name" ,  
> "process\_path" ,"protocol" ,"status" ,"source\_port" ,"destination\_port" ,"direction" ,"cast\_type",  
> "scramble\_state" ,"source\_ip" ,"destination\_ip" ,"sequance\_number" ,"sub\_sequance\_number" ,"user\_name" ,  
> "mog\_counter" ,"destination\_path" ,"reason" ,"dll\_path" ,"dll\_name" ,"chain\_array"  
> ]  
> }  
> mutate {convert =\> ["process\_id","integer"]}  
> mutate {convert =\> ["source\_port","integer"]}  
> mutate {convert =\> ["destination\_port","integer"]}  
> mutate {convert =\> ["sequance\_number","integer"]}  
> mutate {convert =\> ["mog\_counter","integer"]}  
> }  
> output {  
> elasticsearch {  
> hosts =\> "[http://localhost:9200](http://localhost:9200)"  
> index =\> "logs-%{+YYYY.MM.dd}"  
> template =\> "C:\etc\logstash-config\index\_template.json"  
> template\_overwrite =\> "true"  
> }  
> }

curator settings:

```
> actions:
> 1:
> action: delete_indices
> description: >-
> Delete index's older than X days or when reach disk space 200gb
> options:
> ignore_empty_list: True
> filters:
> - filtertype: pattern
> kind: prefix
> value: logs-
> - filtertype: age
> source: name
> direction: older
> timestring: '%Y.%m.%d'
> unit: days
> unit_count: 60
> - filtertype: space
> disk_space: 400
> use_age: True
> source: field_stats
> field: '@timestamp'
> stats_result: max_value
> 
> actions:
> 1:
> action: rollover
> description: >-
> Rollover the index every 1gb.
> options:
> name: logs_write
> conditions:
> max_size: 5g
> max_age: 1d

```

something wrong with my rollover curator - he cant find logs\_Write name aliases  
i added to my elastic template this alias.  
there is an option via logstash to roll every 5gb or every 1day ? (already got every 1 day)

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [January 16, 2019, 2:44pm UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/3 "2019-01-16T14:44:24Z")

</div>

A Rollover alias should be treated different from other aliases. It will _always_ fail if you have the rollover alias name set to be re-added in the index template.

With the Rollover API, you do not need to do this. It takes care of it for you. You create the alias once at initial index creation time, and then rollover handles it for you.

---

<div class="post-metadata">

**Author:** ![bmatoki](https://avatars.discourse-cdn.com/v4/letter/b/7ea924/32.png) [@bmatoki](https://discuss.elastic.co/u/bmatoki)\
**Post date:** [January 16, 2019, 2:58pm UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/4 "2019-01-16T14:58:15Z")

</div>

i tried to add conditions tothe index template but cant start my logstash with this configuration. its failed.

```
  "aliases": {
"logs_write": {}
  },
"conditions": {
"max_size": "5gb"
  },

```

log 🙂

`

> ] Failed to install template. {:message=\>"Got response code '400' contacting Elasticsearch at URL '[http://localhost:9200/\_template/logstash](http://localhost:9200/_template/logstash)'", :class=\>"LogStash::Outputs::Elasticsearch::HttpClient::Pool::BadResponseCodeError", :backtrace=\>["C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/http\_client/manticore\_adapter.rb:80:in `perform_request'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:291:in `perform\_request\_to\_url'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:278:in `block in perform_request'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:373:in `with\_connection'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:277:in `perform_request'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:285:in `block in Pool'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/http\_client.rb:348:in `template_put'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/http_client.rb:86:in `template\_install'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/template\_manager.rb:21:in `install'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/template_manager.rb:9:in `install\_template'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/common.rb:127:in `install_template'", "C:/Cyber20/ELK/logstash-6.5.2/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/common.rb:49:in `block in install\_template\_after\_s

`

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [January 16, 2019, 3:41pm UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/5 "2019-01-16T15:41:37Z")

</div>

Ah, I see. You misunderstand the Rollover API. Those conditions are used when the API is called, not when the alias is created.

The rollover API must be called periodically to do the actual rollover. It's not automatic.

To re-iterate, you should create the index using the REST API first, then just use that in Logstash.

```auto
PUT /logs-000001 
{
  "aliases": {
    "logs_write": {}
  }
}

```

Then you change your Logstash to point to `logs_write`:

```auto
output {
  elasticsearch { 
    hosts => "http://localhost:9200"
    index => "logs_write"
    template => "C:\etc\logstash-config\index_template.json"
    template_overwrite => "true"
  }
}

```

You can continue to use your custom index template, but it cannot contain an `alias` section with the `logs_write` alias. Once created, you never have to touch it again. Rollover takes care of everything for you, but you _must_ call it periodically for the rollover to happen. It's not automatic.

Review the [Rollover API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/6.5/indices-rollover-index.html) if this is unclear.

---

<div class="post-metadata">

**Author:** ![bmatoki](https://avatars.discourse-cdn.com/v4/letter/b/7ea924/32.png) [@bmatoki](https://discuss.elastic.co/u/bmatoki)\
**Post date:** [January 17, 2019, 6:34am UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/6 "2019-01-17T06:34:00Z")

</div>

ok thanks.  
so i changed my index name at logstash config to logs-  
delete the aliases from index template.

added alias

```
POST /_aliases
{
    "actions" : [
        {
            "add" : {
                 "index" : "logs-",
                 "alias" : "test"
            }
        }
    ]
}

```

i saw at [http://localhost:9200/logs-?pretty=true](http://localhost:9200/logs-?pretty=true)  
the alias.

my curator settings is

```
 action: rollover
description: >-
  Rollover the index every 5gb or 1d.
options:
  name: test
  conditions:
    max_age: 1d
    max_docs: 1000000
    max_size: 5gb

```

but i get an error from curator 🙂

> 2019-01-17 08:24:58,847 INFO Preparing Action ID: 2, "rollover"  
> 2019-01-17 08:24:58,863 INFO Trying Action ID: 2, "rollover": Rollover the index every 5gb or 1d.  
> 2019-01-17 08:24:58,863 ERROR Failed to complete action: rollover. \<class 'ValueError'\>: Unable to perform index rollover with alias "test". See previous logs for more details.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [January 17, 2019, 2:16pm UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/7 "2019-01-17T14:16:56Z")

</div>

This is almost certainly because `logs-` is not a viable rollover index. A rollover index name should be a pattern followed by a dash and an incrementable number, like `index-000001`. When a rollover is called, it will increment that number.

There are exceptions to this rule (e.g. using dates in the name), but even those exceptions ought to end with a dash and a number.

---

<div class="post-metadata">

**Author:** ![bmatoki](https://avatars.discourse-cdn.com/v4/letter/b/7ea924/32.png) [@bmatoki](https://discuss.elastic.co/u/bmatoki)\
**Post date:** [January 20, 2019, 5:57am UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/8 "2019-01-20T05:57:16Z")

</div>

ok , now the rollover via curator work thanks.  
its create a new index by the conditions but it still adding docs to the first one.  
logstash;  
index =\> "logs-000001"  
added aliases via route \_aliases with action add.  
the new index called logs-000002 but the docs increase to the first index.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [January 20, 2019, 2:50pm UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/9 "2019-01-20T14:50:39Z")

</div>

It seems likely that your Logstash is shipping to an index name, rather than an alias. For the procedure I shared previously to work, Logstash cannot be shipping to the alias you are creating before it is created, otherwise Logstash will create that alias name as an index name instead.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 17, 2019, 2:50pm UTC](https://discuss.elastic.co/t/cant-roll-over-index-via-index-template/164264/10 "2019-02-17T14:50:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.