Hadoop snapshot repository cannot create


(Gunho Shin) #1

Hi, I'm elasticsearch beginner and development log management system on my local computer's VMs.
I trying snapshot to HDFS but repository was not created on CentOS6.6.

env : Hadoop2.5.2, ES 1.5.1, OpenJDK 1.7_75

  1. install and confirm repository-hdfs plugin(hadoop2) on my cluster(every nodes).

  2. download and copy elasticsearch-hadoop-2.0.2.jar and hadoop-client-2.5.2.jar at $ES_HOME/lib

  3. edit elasticsearch.yml

    repositories.hdfs.uri: "hdfs://192.158.10.251:8020/"
    repositories.hdfs.path: "es/snapshot"
    repositories.hdfs.load_defaults: "true"
    repositories.hdfs.conf_location: ["/home/hadoop/hadoop-2.5.2/etc/hadoop/core-site.xml","/home/hadoop/hadoop-2.5.2/etc/hadoop/hdfs-site.xml"]
    repositories.hdfs.concurrent_streams: 5
    repositories.hdfs.compress: "true"
    repositories.hdfs.chunk_size: "10mb"

  4. curl command run

    curl -XPOST 'http://ES_MASTER_IP:9200/_snapshot/hdfs_repo0' -d
    '{"type":"hdfs","settings":{
    "uri":"hdfs://HDFS_NAMENODE_IP:8020/","path":"es/snapshot"
    }}'

  5. return RemoteTransportException 'failed to create repository'

    {"error":"RemoteTransportException[[ES3][inet[/192.168.10.210:9300]][cluster:admin/repository/put]]; nested: RepositoryException[[hdfs_repo0] failed to create repository]; nested: CreationException[Guice creation errors:\n\n1) Error injecting constructor,
    java.lang.NoClassDefFoundError:
    org/elasticsearch/common/blobstore/ImmutableBlobContainer
    at org.elasticsearch.repositories.hdfs.HdfsRepository.()
    at org.elasticsearch.repositories.hdfs.HdfsRepository
    at Key[type=org.elasticsearch.repositories.Repository, annotation=[none]]
    \n1 error]; nested: NoClassDefFoundError[org/elasticsearch/common/blobstore/ImmutableBlobContainer]; nested: ClassNotFoundException[org.elasticsearch.common.blobstore.ImmutableBlobContainer]; ","status":500}

I search this exception on google but cannot solve. I need help. Thanks.


(Costin Leau) #2

You are using an old version of elasticsearch-hadoop. Or rather, the latest Elasticsearch (1.5.2) has significant internal changes that it no longer works against 2.0.2
Please use 2.1.Beta4 instead


(Gunho Shin) #3

I update my repository-hdfs plugin version to 2.1.0.Beta4-hadoop2 and it works fine. Thanks!


(Costin Leau) #4

Glad to hear it!


(system) #5