ES - Yarn - HDFS or NFS

Elastic Stack Elasticsearch
1

Hi

I am trying to run Elasticsearch in yarn mode. Works fine if I have datapath set to local disk.

I am getting ClassNotFound exception if change the gateway.type to fs or hdfs. From documentation looks like only local gateway module is available and rest are deprecated.

I mounted the hadoop as NFS now I am getting Stale File Handle Exception. Any thoughts ?

2015-10-13 15:54:16,423][DEBUG][transport.netty ] [Shola Inkosi] connected to node [[Ani-Mator][Brt2Uuh1QSG5CHJESOPyyQ][n002][inet[/10.11.13.202:9300]]]
va:77)
at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:245)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
Caused by: java.nio.file.FileSystemException: /mapr/apps/elasticsearch/distdata/data/elasticsearch/nodes/0/es.tmp: Stale file handle
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)

2

Ok Changing the datapath to
/apps/elasticsearch/${HOSTNAME}/data worked.
Now how to solve the restarts/node failures. In Yarn there is no guarantee same nodes are used after the restart. Is there any environment variable to get the nodeID like '0', '1', '2'. Any pointers are appreciated.