Two different partitions in the same elaticsearch server


(Anantha Rao Naidu) #1

Hi Support,

How to manage two different partitions in the same elaticsearch server and both partitions should talk to kibana dashboard.

How to move indexes to the new partition in the same server by using an Elastic search.

Thanks,
Ananth


(Magnus Bäck) #2

When you say "partitions", do you mean cluster partitions (an Elasticsearch term describing a bad situation) or disk partitions? Or are you talking about something else?


(Anantha Rao Naidu) #3

its disk partition.


(Mark Walkom) #4

You can set multiple values in path.data to then use multiple partitions.


(Anantha Rao Naidu) #5

We have tried but it was not working, Please explain the detailed steps to follows.

And could you please let us know, if there is any alternative way.


(Magnus Bäck) #6

Please explain what you tried and we'll take it from there. Include configuration snippets, relevant logs, and explain what "not working" means to you.


(Anantha Rao Naidu) #7

We have followed the below steps.

  1. We have added new file path in elasticsearch.yml at path.data
  2. Moved some logs to new file path.
  3. Restarted Elasticsearch and Kibana services.

(Magnus Bäck) #8

And the logs are actually read by Logstash and added to Elasticsearch? The reason could be that with the default setting of index.store.distributor, files will be stored in the directory with the most free space. You can change this to pick directories randomly. See the directory layout documentation. This behavior will change in ES 2.0; see github.com/elastic/elasticsearch/issue #9498).

Note that ES won't balance the data between the data directories in the same way as shards are balanced between nodes in a cluster. The newly added partition will only be considered for newly created files, and the logs that you just ingested might not have triggered a segment merge that led to new files being created.


(Anantha Rao Naidu) #9

Hi Support,

Could you please let us know, how to do Shards allocation in primary partition and how to move those shards to secondary partition in a single server.

Thanks,
Ananth


(Mark Walkom) #10

You can't.


(Anantha Rao Naidu) #11

thanks, can we create one Shard in Primary Partition and create another Shard in Seconday Partition? If so is there a mechanism/tool to use to copy log files from one Shard to other Shard.


(Mark Walkom) #12

Again, you can't.

You need to run multiple nodes on the one host to handle multiple mounts/partitions of different types (SSD/HDD).


(system) #13