# Issue with Elasticsearch indexes

**URL:** <https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [April 28, 2023, 2:13pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064 "2023-04-28T14:13:23Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![milank2](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@milank2](https://discuss.elastic.co/u/milank2)\
**Post date:** [April 28, 2023, 2:13pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/1 "2023-04-28T14:13:23Z")

</div>

Hello everyone, I am new to ELK and the issue I am experiencing is that indexes are after 3 days reduces to 25x bytes and 0 documents. We are viewing index patterns in Kibana but it will display the 3 days only. Nothing before that.  
We are running Elasticsearch and Kibana 7.12.1 using docker-compose.  
I have setup policy which should keep the data in for at least 10 days but the indexes older then 3 days are reduced to 253bytes and 0 documents  
Can someone help.  
Thank you

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [April 28, 2023, 2:16pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/2 "2023-04-28T14:16:56Z")

</div>

Hello @milank2

Welcome to the elastic community!!!  
Could you share us the [ilm explain](https://www.elastic.co/guide/en/elasticsearch/reference/7.12/ilm-explain-lifecycle.html) output

---

<div class="post-metadata">

**Author:** ![milank2](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@milank2](https://discuss.elastic.co/u/milank2)\
**Post date:** [April 28, 2023, 3:10pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/3 "2023-04-28T15:10:36Z")

</div>

```auto
  "indices" : {
    "msg-stas-audit-panel-state-2023-4-26" : {
      "index" : "msg-stas-audit-panel-state-2023-4-26",
      "managed" : true,
      "policy" : "msg-stas-audit-policy-policy",
      "lifecycle_date_millis" : 1682467139629,
      "age" : "2.62d",
      "phase" : "new",
      "phase_time_millis" : 1682531366167,
      "action" : "complete",
      "action_time_millis" : 1682531366167,
      "step" : "complete",
      "step_time_millis" : 1682531366167,
      "phase_execution" : {
        "policy" : "msg-stas-audit-policy-policy",
        "version" : 1,
        "modified_date_in_millis" : 1682531059835

```

---

<div class="post-metadata">

**Author:** ![milank2](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@milank2](https://discuss.elastic.co/u/milank2)\
**Post date:** [April 28, 2023, 3:11pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/4 "2023-04-28T15:11:08Z")

</div>

Thanks for replying here it is

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 28, 2023, 4:14pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/5 "2023-04-28T16:14:31Z")

</div>

Hi @milank2 Welcome to the community.

Can you run

`GET _cat/indices?v`

> [@milank2](#):
>
> I am experiencing is that indexes are after 3 days reduces to 25x bytes and 0 documents.

Something else is going on as there is no ILM action that does this.

Those 0 document 255 byte indices are not "deleted" or cleaned indices they are "stub" indices created by a rollover policy and then data is never written to them.

Please also share the

`GET _ilm/policy/msg-stas-audit-policy-policy`

---

<div class="post-metadata">

**Author:** ![milank2](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@milank2](https://discuss.elastic.co/u/milank2)\
**Post date:** [April 28, 2023, 5:00pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/6 "2023-04-28T17:00:50Z")

</div>

Hi Stephenb, here it is:  
for GET /\_cat/indices?v I only took msg-stas\* since it is so big:

```auto
GET /_cat/indices/msg-stas-audit-*?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open msg-stas-audit-asdr-message-2023-4-25 X5ZQbbN5Q06i0BgHGYA5hA 1 0 0 0 253b 253b
green open msg-stas-audit-panel-state-2023-4-28 5zsirpTzT16jXH2MAu03yg 1 0 554217 0 168.2mb 168.2mb
green open msg-stas-audit-asdr-message-2023-4-24 -qg11lz9TMur6itk_Jy3aQ 1 0 0 0 253b 253b
green open msg-stas-audit-asdr-message-2023-4-23 1GPreZGzSJC2Hf3weI34tQ 1 0 0 0 253b 253b
green open msg-stas-audit-asdr-message-2023-4-22 TLQkFIY8TC-3R30GSPWMog 1 0 0 0 253b 253b
green open msg-stas-audit-asdr-message-2023-4-21 B22fHdhTTAOXcsVCicyToA 1 0 0 0 253b 253b
green open msg-stas-audit-asdr-message-2023-4-20 xBPpf7JgQaSoMHfCOuXQFA 1 0 0 0 253b 253b
green open msg-stas-audit-panel-state-2023-4-25 YXFfV-vGTKqf6UYweQZxZg 1 0 0 0 250b 250b
yellow open msg-stas-audit-asdr-message-2023-4-28 jVzqwYZQTuGn-cILhGAlyQ 1 1 4913624 0 354.6mb 354.6mb
yellow open msg-stas-audit-asdr-message-2023-4-27 tMNvpWaVStquVhvQACtXiA 1 1 4327796 0 306mb 306mb
yellow open msg-stas-audit-panel-state-2023-4-27 PN-qgRcXSWCcED3ZtRd3dQ 1 1 489735 0 142.6mb 142.6mb
green open msg-stas-audit-asdr-message-2023-4-26 NoZpcGrVTNajl8MrT4wwCg 1 0 6846149 60 483.7mb 483.7mb
green open msg-stas-audit-panel-state-2023-4-26 I7KIUOvFTrOgywbjBsAKdg 1 0 714810 6 207.4mb 207.4mb
green open msg-stas-audit-panel-state-2023-1-24 emquHpEySj2Somba5GCw-Q 1 0 0 0 231b 231b

```

and

```auto
  "msg-stas-audit-policy-policy" : {
    "version" : 12,
    "modified_date" : "2023-04-28T14:21:44.028Z",
    "policy" : {
      "phases" : {
        "warm" : {
          "min_age" : "15d",
          "actions" : { }
        },
        "cold" : {
          "min_age" : "30d",
          "actions" : {
            "readonly" : { }
          }
        },
        "hot" : {
          "min_age" : "10d",
          "actions" : {
            "rollover" : {
              "max_size" : "5gb",
              "max_age" : "14d"
            }
          }
        },
        "delete" : {
          "min_age" : "60d",
          "actions" : {
            "delete" : {
              "delete_searchable_snapshot" : true
            }
          }
        }
      }
    }
  }

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 28, 2023, 5:18pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/7 "2023-04-28T17:18:41Z")

</div>

Hi @milank2

> [@milank2](#):
>
> ```auto
> "hot" : {
> "min_age" : "10d",
> 
> ```

1st that is not correct... that should be

```auto
        "min_age": "0ms",

```

That could be the source of your problem... but not sure

```auto
green open msg-stas-audit-asdr-message-2023-4-24 -qg11lz9TMur6itk_Jy3aQ 1 0 0 0 253b 253b
green open msg-stas-audit-asdr-message-2023-4-23 1GPreZGzSJC2Hf3weI34tQ 1 0 0 0 253b 253b
green open msg-stas-audit-asdr-message-2023-4-22 TLQkFIY8TC-3R30GSPWMog 1 0 0 0 253b 253b
green open msg-stas-audit-asdr-message-2023-4-21 B22fHdhTTAOXcsVCicyToA 1 0 0 0 253b 253b
green open msg-stas-audit-asdr-message-2023-4-20 xBPpf7JgQaSoMHfCOuXQFA 1 0 0 0 253b 253b

```

Those are created by rollover ... interesting thing is that they have 0 Replicas which make me thing you have another index template that might be matching

First fix the ILM policy ... clean up those empty indices and then observe

---

<div class="post-metadata">

**Author:** ![milank2](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@milank2](https://discuss.elastic.co/u/milank2)\
**Post date:** [April 30, 2023, 9:08am UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/8 "2023-04-30T09:08:57Z")

</div>

Hi Stephenb.  
Thanks for replying, so I have changed the policy to "min\_age": "0ms"  
removed most of the indexes with 0 docs and 25x bytes. Interesting thing was that I have recovered bunch of shards by doing that.  
Anyhow Kibana was still displaying only last 3 days, well seeing still the same results I have removed all the policy's and aliases associated with the indexes but the 3 day display in Kibana still remains.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 30, 2023, 1:38pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/9 "2023-04-30T13:38:14Z")

</div>

> [@milank2](#):
>
> Interesting thing was that I have recovered bunch of shards by doing that.

You may have some other issues... Keep an eye out

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 30, 2023, 10:56pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/10 "2023-04-30T22:56:38Z")

</div>

Welcome to our community! 😃

> [@milank2](#):
>
> We are running Elasticsearch and Kibana 7.12.1

Please note that version is [EOL](https://www.elastic.co/support/eol) and no longer supported, you should be looking to upgrade as a matter of urgency.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 28, 2023, 10:57pm UTC](https://discuss.elastic.co/t/issue-with-elasticsearch-indexes/331064/11 "2023-05-28T22:57:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.