# Index rollover latency

**URL:** <https://discuss.elastic.co/t/index-rollover-latency/222866>\
**Category:** Elasticsearch\
**Created:** [March 10, 2020, 6:24am UTC](https://discuss.elastic.co/t/index-rollover-latency/222866 "2020-03-10T06:24:41Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Muhammad\_Ariful\_Isla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muhammad_ariful_isla/32/55732_2.png) [@Muhammad\_Ariful\_Isla](https://discuss.elastic.co/u/Muhammad_Ariful_Isla)\
**Post date:** [March 10, 2020, 6:24am UTC](https://discuss.elastic.co/t/index-rollover-latency/222866/1 "2020-03-10T06:24:41Z")

</div>

I have created an `ILM` policy as follows,

```auto
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_docs": 30
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "15m",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

```

Index setting as follows,

```auto
{
  "index": {
    "lifecycle": {
      "name": "index-roll+doc-over",
      "rollover_alias": "router01"
    },
    "number_of_shards": "1",
    "number_of_replicas": "0"
  }
}

```

As you have already guessed, I am testing this ILM rollover policy which is why I have set the document size only `30`. I am planning to limit the doc size of 1.5 Billion. But I am observing huge latency while creating a rollover index and switching to the new index. It is writing on the current index and not rolling over. For the test I have ingested only `500` lines of log.

Index details

```auto
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

green open router01-000001 654G2ArFQiypDn9U69jedw 1 0 500 0 151.1kb 151.1kb

```

And at the time of writing this question, the other index (`router01-000001`) is not created yet.

The the `_ilm/explain` is following,

```auto
{
  "indices" : {
    "router01-000001" : {
      "index" : "router01-000001",
      "managed" : true,
      "policy" : "index-roll+doc-over",
      "lifecycle_date_millis" : 1583819391191,
      "age" : "14.08m",
      "phase" : "hot",
      "phase_time_millis" : 1583819391396,
      "action" : "rollover",
      "action_time_millis" : 1583819706013,
      "step" : "check-rollover-ready",
      "step_time_millis" : 1583819706013,
      "phase_execution" : {
        "policy" : "index-roll+doc-over",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_docs" : 30
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "version" : 1,
        "modified_date_in_millis" : 1583756555777
      }
    }
  }
}

```

Elasticsearch currently has `2GB` heap and logstash has 1GB. Total Memory 4GB.  
{  
UPDATE:

- I have start ingesting larger file and after storing `15500` docs it rolled over to new index.

My question is,

- Is the latency expected (May be I misunderstood the rollover concept)
- Did I mistake anywhere?
- If not than what can I do to improve the latency as there will be 20 millions log will be ingested per day

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 7, 2020, 6:24am UTC](https://discuss.elastic.co/t/index-rollover-latency/222866/2 "2020-04-07T06:24:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.