Cat allocation disk.indices higher than disk.used or disk.total

Elastic Stack Elasticsearch
1

Hello,

We have a cluster that was upgraded priori this week from version 5.6.16 to version 6.8.13 and upgrade succeeded and everything was running as expected.

One day later one of the data node reached disk watermark (low then high) and cluster started relocating but we soon ran to flood_stage watermark.
Trying to troubleshoot this issue we checked disk allocation and result was puzzling to me :

shards disk.indices disk.used disk.avail disk.total disk.percent node
   914      367.8gb     293gb     26.4gb    319.5gb           91 xxx-2
   941      344.4gb   281.5gb     10.6gb    292.1gb           96 xxx-3
   617      239.6gb   194.3gb      5.7gb    200.1gb           97 xxx-1

To my understanding, disk.total is available volume, disk used is global usage (eg ES + rest of the world) disk.indices is usage by ES (that i would expect be a part of disk.used)

Can someone see what I misunderstood or tell me if this may be a known issue ?

My assumption on the watermark that were reached is not that data are growing but more that free space is eroding.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.