Elasticsearch on google cloud storage

Elastic Stack Elasticsearch
1

Hi,

We have a es cluster of 40 nodes running in google cloud . We have 150 tb of SSD allocated to this cluster.
But we are trying to move our storage from SSD to google cloud storage buckets.
I have mounted a path in my data nodes to Google cloud storage buckets and have give that path in data paths of each data node. ES cluster it up but when i stream data to the cluster master nodes throws errors that data nodes are not available. I have few questions on this approach
Is using Google cloud storage for data nodes a recommended approach?
If so what is the better way of doing it

2

Bad idea IMO. Read:

Always use local storage, remote filesystems such as NFS or SMB should be avoided. Also beware of virtualized storage such as Amazon’s Elastic Block Storage. Virtualized storage works very well with Elasticsearch, and it is appealing since it is so fast and simple to set up, but it is also unfortunately inherently slower on an ongoing basis when compared to dedicated local storage. If you put an index on EBS, be sure to use provisioned IOPS otherwise operations could be quickly throttled.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.