Best option for persistent volume of an on-premise ECK

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

We have a cluster with multiple elastic instances deployed on a few servers. We decided to use ECK to manage our cluster. We are looking for the best option for persistent volume for an ECK cluster.

We have tried NFS and Local Volumes. However, NFS volume is not recommended to be used with Elasticsearch (here and here). We also have problems using Local volume (Pod crashes after a few document insertion).

Could you please help us to choose best volume option for our case?

We use Debian on our servers.

2

Hi @Ali_Reza_Haghighat,

You're right: we do not recommend NFS, which generally offers poor performance for Elasticsearch.

We also have problems using Local volume (Pod crashes after a few document insertion)
This seems a bit strange. How are you handling the local volumes?

A few options you may want to look at:

  • If you have any network-attached PersistentVolume provider available, that's by far the easiest option. GCP persistent disks and AWS ELBs now offer pretty good performance, depending on which kind of volume you select of course.
  • If you only have local disks available, you can look into local PersistentVolumes:
    • you can create them manually or use the static provisioner to pre-create the PV resources according to the nodes existing disks. In both case the storage class of those volumes should match the storage class of the volumeClaimTemplates in your Elasticsearch specification.
    • Rancher local path provisioner provisions hostPath on the nodes dynamically.
    • TopoLVM is an advanced local volume provisioner which integrates with LVM. It offers way more features than the static provisioner.
    • There are other commercial local PersistentVolume provider offering out there.
1 Like