What is the best way to distribute nodes and shards in Elasticsearch to achieve fast search while storing recent data on SSD and older data on HDD?

Elastic Stack Elasticsearch
21

Okay, I will follow your advice.
Thank you both for your help and for the quick response

22

Great advice from
@Christian_Dahlqvist I add only to confirm that server will be 100% dedicated to elasticsearch ? Your initial plan only allocated 320GB of it for 5x hot nodes?

And second, 75TB of local SSD storage is a lot. But equally important here is the aggregate IOps/bandwidth you can realistically achieve. Best if you have as many independent paths to the storage as VMs.

Lastly, single server solution is obviously no redundancy. Consider you “server on fire” scenarios.

1 Like
23

Yes, initially this will be 320 GB, but what do you recommend? I can increase it to 128 GB per node, or even divide the 1.5 TB among the five nodes.

The server has 24 drives of type KR-09N32F-SSW00-468-00LM-A02 — you can look into this.

Yes, you are correct; if the entire server burns out or fails, I have a DR plan in place.

24

I recommend giving almost all the memory to elasticsearch VMs. @Christian_Dahlqvist suggested starting to test at 10x, and I’ve no reason to argue with that. How useful is leaving memory unallocated, letting host OS use it for caching? My hunch is “not very”.

On the disks , you need know about the paths to the disks. 24 disks are across how many controllers? Actually, there 101 ways you could map your dish’s to VMs. Does your host and VM OS need to live on same disks? If so, maybe assign 2 for that, which leaves 22, which is 2x11. 24 is a bit awkward to divide by 10.

25

I have SSD/NVME on proxmox and if I use ceph or even map drive via proxmox to vm it is slow, very slow.

After lot of testing I have found that only direct pass through disk to VM works best and gets me speed that is describe by vendor.

this means you can’t fail over vm. for example one of the VM. and scsi2 is my data disk, scsi0 is OS

image
image1083×278 14.4 KB

26

There is only one controller, and all the data functions as a single hard disk. I used RAID 6 for this setup.

27

In my case, there are no physical machines. I am creating virtual machines from the server, dividing it into 5 virtual machines.

28

then nothing much I guess you can do. run some benchmarking for example run dd for read and write

one dd, then 5 dd and then may be 20 and see average speed it does

for example:

dd if=/dev/random of=/datafilesstem/ella1.test count=10000000