Elasticsearch data directory in S3 bucket

Elastic Stack Elasticsearch
1

Hi All,

Is it possible to have the data directory of a newly built ES 8 cluster hosted on a S3 bucket.

Idea is for the data nodes to use S3 instead of local disk or NAS.

Thanks

2

No, this is not possible.

3

Thanks for a quick response.

We have a retention period of 30 days for the indices.

Can we move indices older than 7 days to S3 and how? Idea is to have more disk space available on local.

We have a cluster of 8 nodes with 5 data and 3 dedicated master nodes and indices are written into local disk on each data node.

Thanks

4

You can achieve this by configuring searchable snapshots in a hot-cold architecture. This does however require a commercial Enterprise license. You can not have searchable data on S3 with the free basic license.

5

Thanks. I have a very basic question.

In our case, all the data nodes are writing to their local disks and there is no common storage (e.g NAS) between the data nodes.

Can snapshots be taken in such set ups?

6

Snapshots require shared storage, e.g NFS mount or S3.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.