CEPH Storage For Elasticsearch

[Ankita_Pachauri]

We are planning to create an Elasticsearch cluster over Kubernetes, and for this CEPH distributed file system would be used as Storage, I understand that elastic recommends using local storage, however are there any known performance issues or other issues in general using CEPH. Has anyone from elastic tested Elasticsearch 7 on CEPH storage?

[Christian_Dahlqvist]

I did a quick search and found the following reasonably recent threads:

Elasticsearch support for Distributed FS like Rook-ceph

Hi, Does Elasticsearch architecture supports Distributed File system for storage like rook-ceph?

Is ElasticSearch supports CephRBD or CephRBD-fast with RWO access?

Is ElasticSearch supports CephRBD or CephRBD-fast with RWO access ?

Rook ceph for Elastic Search

Hello, We are deploying elastic search on kubernetes farm and we have rook ceph as our distributed storage. I was going through earlier topic and there was enough reason than with ES being heavy used for File system. But, is there a plan at this point to support distributed storage? what is the recommendation for kubernetes based deployments with distributed storage ?

Elasticsearch IOPS

Hello Team I am having Elasticsearch version 5.6.3 running on RHEL 7.4. This Cluster is of 10 nodes. Hardware : 8 CPU and 32 GB RAM. Hard Disk : HDD { CEPH Storage } Elastic Data Paths : Multiple { 7 Different mount points }. We are logging logs data into this ES cluster. When we are performing IO Test using "fio" tool , we are getting throughput of sequential write to multiple mount points as below WRITE: bw=20.3MiB/s (21.3MB/s), 20.3MiB/s-20.3MiB/s (21.3MB/s-21.3MB/s), io=24.0GiB (25.8G…

I am not aware of any recent changes that would change the recommendations in these posts.

[tusharnemade]

On CEPH it works .. but its quite slow. You might face multiple Timeout Issues for internal operations of Elasticsearch and some of your own operations you would be performing on Elasticsearch Cluster.

You might split the roles of Nodes ... to have less overhead on actual data nodes ...

[Ankita_Pachauri]

For licensed elasticsearch cluster, would the elasticsearch support team help with issues arising on cluster running ceph storage?
Can someone from elasticsearch answer this?

[DavidTurner]

It sort of depends what you mean by "support". Everyone gets the same level of help, which includes assistance with diagnosing performance issues, but if the diagnosis determines your storage is too slow for your needs (or behaves incorrectly under load) then that's something you would be responsible for fixing.

Distributed storage is slower than local storage because that's how physics works unfortunately, it's not something you can work around with a support contract.