Move default elastic data/index storage to a newly created volume

Elastic Stack Elasticsearch
1

We have successfully deployed ELK and it's up and running. We've created a new volume and attached it to the VM hosting the ELK stack. Our goal is to store all data, including indices, on this newly mounted volume.

To do this, we followed the steps and modified the path.data value in /etc/elasticsearch/elasticsearch.yml to point to the new volume's path. However, after restarting Elasticsearch, the service failed to start, and we were unable to log into Kibana.

I reverted the changes to the default path, and everything is back to normal. Can someone guide me on the correct steps to configure Elasticsearch to store data on the new volume?

2

Hi

Few Qs

we followed the steps

Which steps do you mean here?

Did you sync/copy the data from old path to new path? (rsync is great tool for this)

Did you check permissions? e.g. if elasticsearch is running as user elastic, it will need read/write permissions to the new directory tree.

store all data, including indices, on this newly mounted volume

This is fine, but what data, aside from the indices, did you mean?

after restarting Elasticsearch, the service failed to start

You changed the elasticsearch.yml file while elastic was running? Anyways, always helpful to post the actual log when something goes wrong, please do so here.

And lastly, just to confirm, we are talking about a single-node elasticsearch instance here, not part of a larger cluster, right?

Good news is what you are trying to do is pretty straightforward, so I expect we can sort this out quite quickly, likely you just missed something or there is a slight misunderstanding we can correct.

3

Thank you Rain I followed this step and this time elastic is up and running, let me monitor it and get back to you, I did not remove the old data, I just copied it to the new location. let me know if you have any comment on my step:

Step I followed:
- Created the new volume and it is been mounted, /aap_automation:
[root@AskAI-Worker2 ~]# df -kh
Filesystem Size Used Avail Use% Mounted on
devtmpfs 16G 0 16G 0% /dev
tmpfs 16G 168K 16G 1% /dev/shm
tmpfs 16G 394M 16G 3% /run
tmpfs 16G 0 16G 0% /sys/fs/cgroup
/dev/vda3 320G 61G 260G 19% /
/dev/vda2 100M 5.8M 95M 6% /boot/efi
tmpfs 3.2G 0 3.2G 0% /run/user/1000
/dev/vdb 984G 398M 933G 1% /aap_automation

- Stop elastic search
systemctl stop elasticsearch

- Cp elasticsearch folder with all its permission:
cp -rp /var/lib/elasticsearch/ /aap_automation/

- Change path for data to the new location:
vi /etc/elasticsearch/elasticsearch.yml
	○ Change: path.data: /aap_automation/elasticsearch
- Start elastic search:

systemctl start elasticsearch

4

Do not EVER delete or modify data directly within the Elasticsearch data path as that can lead to complete data loss. Always remove data using the Elasticsearch APIs.

5

To be honest, that looks like the right sequence. Personally I would have used rsync, but ... probably doesn't really matter if the cp completed (rsync is better if a big copy gets interrupted, as it will just pick up from where it left off)

I am not sure what you did wrong? You quoted me quoting you with:

and added:

Thank you Rain I followed this step and this time elastic is up and running

Mmm, not completely clear what you meant there.

Anyways, if its now up and running then I think you are good.

@Christian_Dahlqvist I saw your comment too, which is obviously good advice, but I believe @ermisma just copied the directory while elasticsearch was NOT running, then changed path.data in elasticsearch.yml, then started ES again, so that should be OK.

1 Like
6

Thanks you again. I am good

7

Er, something again not right ?

8

My apology, earlier I meant I am now good. Thank you for your help. It was very helpful.

1 Like