ElasticSearch Hadoop Repository issue

In out production environment we are having Cloudera distributed Hadoop version 2.6.0-cdh5.11.0 and our Elastic search version is 2.4.1.
If I install repository-hdfs plugin , I'm getting hadoop-2.7.1 And it ends in hadoop version conflicts.
So I replaced all the jars in hadoop-libs to the relevant Hadoop version in my production environment (i.e 2.6.0-cdh5.11.0) .
Now while starting Elastic search its showing error like ,
Error -1 ;
Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[2017-08-28 09:28:43,201][WARN ][repositories ] [Plasma] failed to create repository [hdfs][hadoop]
org.elasticsearch.common.inject.CreationException: Guice creation errors:

  1. Error injecting constructor, ElasticsearchGenerationException[Cannot create Hdfs file-system for uri [hdfs://52.220.175.57:8020/]]; nested: HadoopIllegalArgumentException[The short-circuit local reads feature is enabled but dfs.domain.socket.path is not set.];
    at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source)
    while locating org.elasticsearch.repositories.hdfs.HdfsRepository
    while locating org.elasticsearch.repositories.Repository

Kindly help me out...

The error message contains some information that would be a good starting point. It seems that the client has short-circuit local reads enabled in the settings, but the setting dfs.domain.socket.path is not set, thus throwing an error. Are you setting short-circuit local reads to enabled in the repository settings? How are you loading your configurations?

Ya we have enabled the property "dfs.domain.socket.path " and assigned a path as well. Eventhough its showing eerror.

Could you include your HDFS repository settings here? Is this property required for you? Are you hosting HDFS nodes and ES nodes on the same hardware?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.