ElasticSearch multiple path.data not storing in all directories

I have 3 nodes of ElasticSearch v5.5 (node0, node1, node2)

I have set path.data to 5 different directories for each node on CentOS 7
i.e. /elk_data0, /elk_data1, /elk_data2, /elk_data3, /elk_data4

My problem is data is not getting stored on /elk_data0

The size of /elk_data0 is only 33MB and for other directories it is above 25GB

I found the below difference in /elk_data0 with others

/elk_data0/es_data/node0/nodes/0/indices/1F8ljDHPQcunjWjVfnACnw
In above directory /0/index/ is missing

/elk_data1/es_data/node0/nodes/0/indices/1F8ljDHPQcunjWjVfnACnw/0/index
All directories from 1 - 4 is like the above directory

Each shard is stored in a single path. Do you have enough shards to cover all paths?

Thanks, I have currently 21 indices and 200 shards, but nothing is stored on /elk_data0/ as compared other directory size 33 MB vs 25GB

Did you start out with this disk configuration before adding data to the cluster or change after data was already indexed? Is this imbalance causing any performance problems?

Thanks for your reply, now got it.

I started with 5 directories but for the first directory i had minor typing error in the path,

later i corrected it but till the time data was indexed, I believe sooner or later this directory with get data.

And I didn't faced any performance issue yet.

I don't believe Elasticsearch will move shards between paths once they are placed, so if the path was initially incorrect, it would expect the imbalance. I would expect this to even out as you add additional indices/shards.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.