# Filebeat ILM duplicating index when rolling from hot to warm

**URL:** <https://discuss.elastic.co/t/filebeat-ilm-duplicating-index-when-rolling-from-hot-to-warm/207719>\
**Category:** Elasticsearch\
**Created:** [November 13, 2019, 2:17pm UTC](https://discuss.elastic.co/t/filebeat-ilm-duplicating-index-when-rolling-from-hot-to-warm/207719 "2019-11-13T14:17:12Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\
**Post date:** [November 13, 2019, 2:17pm UTC](https://discuss.elastic.co/t/filebeat-ilm-duplicating-index-when-rolling-from-hot-to-warm/207719/1 "2019-11-13T14:17:12Z")

</div>

We are shipping server events using filebeat and metrics using metricbeat to our Elastic Cloud cluster that has a hot server and a warm server. The cluster is set up with no redundancy, ie just a single hot and a single warm node.

Both metricbeat and filebeat have the same ILM profile - that is to roll over from hot to warm at 40GB. Metricbeat and filebeat are both rolling over OK, but for some reason the filebeat index gets duplicated and the storage becomes 80G when on warm. I also see the status as "green" which usually means that the index is replicated.

How do I prevent the filebeat index from replicating when it rolls over?

 ![31](https://us1.discourse-cdn.com/elastic/original/3X/b/2/b2e06e59a0974442fea6e2e75b5a13dba3e80d7c.png) ![35](https://us1.discourse-cdn.com/elastic/original/3X/9/e/9ebef12749fc1e8e177509e9b19cfe571eff98d5.png)

---

<div class="post-metadata">

**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\
**Post date:** [November 13, 2019, 3:07pm UTC](https://discuss.elastic.co/t/filebeat-ilm-duplicating-index-when-rolling-from-hot-to-warm/207719/2 "2019-11-13T15:07:13Z")

</div>

Moving this to ES since I believe most relevant configuration uses ES directly with no cloud intervention (also this is the public support for the cloud infrastructure, which is deployable separately as "ECE", not the service itself)

---

<div class="post-metadata">

**Author:** ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\
**Post date:** [November 13, 2019, 3:11pm UTC](https://discuss.elastic.co/t/filebeat-ilm-duplicating-index-when-rolling-from-hot-to-warm/207719/3 "2019-11-13T15:11:10Z")

</div>

Thanks, just bizarre that ILM is assigning a replicate to the filebeat index when rolling over, whereas metricbeat is rolling over as expected as I don't have any replica nodes in my cloud deployment.

---

<div class="post-metadata">

**Author:** ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\
**Post date:** [November 13, 2019, 4:39pm UTC](https://discuss.elastic.co/t/filebeat-ilm-duplicating-index-when-rolling-from-hot-to-warm/207719/4 "2019-11-13T16:39:19Z")

</div>

OK, looking at the rolled over indexes closer, it seems like there are two shards being provisioned for the filebeat index and only one for the metricbeat one. I'm still not sure why the total is going to 80GB when rolling over from hot to warm?

Filebeat:

```auto
    {
      "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
      },
      "stats": {
        "uuid": "trI2LaqtQ92gIjm8zBglPw",
        "primaries": {
          "docs": {
            "count": 152068325,
            "deleted": 0
          },
          "store": {
            "size_in_bytes": 43264273311
          },
    "total": {
          "docs": {
            "count": 304136650,
            "deleted": 0
          },
          "store": {
            "size_in_bytes": 86528546619

```

Metricbeat:

```auto
{
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  },
  "stats": {
    "uuid": "9o3tcrG_SW2pjr6rp7nU-Q",
    "primaries": {
      "docs": {
        "count": 43300277,
        "deleted": 16
      },
      "store": {
        "size_in_bytes": 42972823831
      },
"total": {
      "docs": {
        "count": 43300277,
        "deleted": 16
      },
      "store": {
        "size_in_bytes": 42972823831
      },

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 27, 2019, 4:39pm UTC](https://discuss.elastic.co/t/filebeat-ilm-duplicating-index-when-rolling-from-hot-to-warm/207719/5 "2019-11-27T16:39:20Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.