I spend a couple of time to find out how ES can possibly integrate to HDFS.
We have an ES cluster running on top of YARN and want the cluster to be fail safe, e.g. survive a YARN restart.
My conclusion is:
- (1) you can mount HDFS as NFS and point ES to a NFS path (downside: slowdown)
- (2) you can use repository-hdfs and 'manually' care about backup and restore to and from HDFS
Any other options ?
Also i'm yet un-decided on whether to use ES 1.x or 2.x, does it matter in that perspective ?