# Issues with using repository-hdfs plug in for snapshot/restore operation **URL:** \ **Category:** Elasticsearch\ **Created:** [July 7, 2014, 9:42pm UTC](https://discuss.elastic.co/t/issues-with-using-repository-hdfs-plug-in-for-snapshot-restore-operation/18513 "2014-07-07T21:42:04Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Jinyuan\_Zhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinyuan_zhou/32/90185_2.png) [@Jinyuan\_Zhou](https://discuss.elastic.co/u/Jinyuan_Zhou)\ **Post date:** [July 7, 2014, 9:42pm UTC](https://discuss.elastic.co/t/issues-with-using-repository-hdfs-plug-in-for-snapshot-restore-operation/18513/1 "2014-07-07T21:42:04Z")
I am using elasticsearch 1.2.1 and CDH 4.6. quick start vm. My ES server is installed on the same vm. I have one successful senario: I used light version and add the result and command `hadoop classpath` to ES\_CLASSPATH But I encoutered errros with the default version and hadoop2 version. Here is the details of issues. #1. I installed the plugin with this command bin/plugin --install elasticsearch/elasticsearch-repository-hdfs/2.0.0 and I sent a PUT request below: url: [http://localhost:9200/\_snapshot/hdfs\_repo](http://localhost:9200/_snapshot/hdfs_repo) data :{ "type":"hdfs", "settings": { "uri":"hdfs://localhost.localdomain:8020", "path":"/user/cloudera/es\_snapshot" } } I got this response 1. "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: 1. Error injecting constructor, org.elasticsearch.ElasticsearchGenerationException: Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020] at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) while locating org.elasticsearch.repositories.hdfs.HdfsRepository while locating org.elasticsearch.repositories.Repository 1 error]; nested: ElasticsearchGenerationException[Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020]]; nested: RemoteException[Server IPC version 7 cannot communicate with client version 4]; ", 2. "status": 500 I noticed RemoteException: Server IPC version 7 cannot communicate with client version 4 #2 Then I tried hadoop2 version,. So I installed plugin with this command bin/plugin --install elasticsearch/elasticsearch-repository-hdfs/2.0.0-hadoop2 I sent a PUT request as above, this time I even got more strange exectiopon NoClassDefFoundError[org/apache/commons/cli/ParseException] Here is the response. { "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: 1. Error injecting constructor, java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) while locating org.elasticsearch.repositories.hdfs.HdfsRepository while locating org.elasticsearch.repositories.Repository 1 error]; nested: NoClassDefFoundError[org/apache/commons/cli/ParseException]; nested: ClassNotFoundException[org.apache.commons.cli.ParseException]; ", "status": 500 } I wonder if any one have simiar experiences. Note the failed cases are actaully more realistic deplyoment choices. Because my hadoop cluster will less likely be on the same node as my ES server. Thanks, Jack -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/acb15aff-299b-4e4b-bf20-b0ed5a891f60%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/acb15aff-299b-4e4b-bf20-b0ed5a891f60%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Arun\_Prakash](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@Arun\_Prakash](https://discuss.elastic.co/u/Arun_Prakash)\ **Post date:** [October 1, 2014, 4:33am UTC](https://discuss.elastic.co/t/issues-with-using-repository-hdfs-plug-in-for-snapshot-restore-operation/18513/2 "2014-10-01T04:33:46Z")
Hi Jinyuan Zhou, I am also having the same issue... On Tuesday, July 8, 2014 3:12:04 AM UTC+5:30, Jinyuan Zhou wrote: > I am using elasticsearch 1.2.1 and CDH 4.6. quick start vm. My ES server > is installed on the same vm. > I have one successful senario: I used light version and add the result and > command `hadoop classpath` to ES\_CLASSPATH > > But I encoutered errros with the default version and hadoop2 version. > Here is the details of issues. > #1. I installed the plugin with this command > bin/plugin --install elasticsearch/elasticsearch-repository-hdfs/2.0.0 > and I sent a PUT request below: > url: [http://localhost:9200/\_snapshot/hdfs\_repo](http://localhost:9200/_snapshot/hdfs_repo) > data :{ > "type":"hdfs", > "settings": > { > "uri":"hdfs://localhost.localdomain:8020", > "path":"/user/cloudera/es\_snapshot" > } > } > > I got this response > > 1. "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: > > 1. Error injecting constructor, org.elasticsearch.ElasticsearchGenerationException: Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020] > at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) > while locating org.elasticsearch.repositories.hdfs.HdfsRepository > while locating org.elasticsearch.repositories.Repository > > 1 error]; nested: ElasticsearchGenerationException[Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020]]; nested: RemoteException[Server IPC version 7 cannot communicate with client version 4]; ", > 2. "status": 500 > > I noticed RemoteException: Server IPC version 7 cannot communicate with > client version 4 > > #2 Then I tried hadoop2 version,. So I installed plugin with this command > bin/plugin --install > elasticsearch/elasticsearch-repository-hdfs/2.0.0-hadoop2 > > I sent a PUT request as above, this time I even got more strange > exectiopon > > NoClassDefFoundError[org/apache/commons/cli/ParseException] > Here is the response. > > { > "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: > > 1. Error injecting constructor, java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException > at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) > while locating org.elasticsearch.repositories.hdfs.HdfsRepository > while locating org.elasticsearch.repositories.Repository > > 1 error]; nested: NoClassDefFoundError[org/apache/commons/cli/ParseException]; nested: ClassNotFoundException[org.apache.commons.cli.ParseException]; ", > "status": 500 > } > > I wonder if any one have simiar experiences. Note the failed cases are actaully > more realistic deplyoment choices. Because my hadoop cluster will less likely be on the same node > as my ES server. > Thanks, > Jack -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2295d772-931a-4662-a3ea-509fcbac89a7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2295d772-931a-4662-a3ea-509fcbac89a7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Brent\_Halsey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brent_halsey/32/1237_2.png) [@Brent\_Halsey](https://discuss.elastic.co/u/Brent_Halsey)\ **Post date:** [October 2, 2014, 3:17pm UTC](https://discuss.elastic.co/t/issues-with-using-repository-hdfs-plug-in-for-snapshot-restore-operation/18513/3 "2014-10-02T15:17:57Z")
We had the same issue. We're also running CDH 4.6, which expects a different hadoop client. We fixed these by grabbing the source, removing the 'exclude module: "commons-cli"' from repository-hdfs/build.gradle, setting 'hadoop2Version = 2.0.0-cdh4.6.0' in gradle.properties (and set the esVersion and luceneVersions for good measure), and building our own zip: cd repository-hdfs/; ../gradlew -Pdistro=hadoopYarn distZip You can install this with: bin/plugin -u file:////elasticsearch-repository-hdfs-2.1.0.BUILD-SNAPSHOT-hadoop2.zip -i elasticsearch-repository-hdfs Hope this helps. -brent On Monday, July 7, 2014 3:42:04 PM UTC-6, Jinyuan Zhou wrote: > I am using elasticsearch 1.2.1 and CDH 4.6. quick start vm. My ES server > is installed on the same vm. > I have one successful senario: I used light version and add the result and > command `hadoop classpath` to ES\_CLASSPATH > > But I encoutered errros with the default version and hadoop2 version. > Here is the details of issues. > #1. I installed the plugin with this command > bin/plugin --install elasticsearch/elasticsearch-repository-hdfs/2.0.0 > and I sent a PUT request below: > url: [http://localhost:9200/\_snapshot/hdfs\_repo](http://localhost:9200/_snapshot/hdfs_repo) > data :{ > "type":"hdfs", > "settings": > { > "uri":"hdfs://localhost.localdomain:8020", > "path":"/user/cloudera/es\_snapshot" > } > } > > I got this response > > 1. "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: > > 1. Error injecting constructor, org.elasticsearch.ElasticsearchGenerationException: Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020] > at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) > while locating org.elasticsearch.repositories.hdfs.HdfsRepository > while locating org.elasticsearch.repositories.Repository > > 1 error]; nested: ElasticsearchGenerationException[Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020]]; nested: RemoteException[Server IPC version 7 cannot communicate with client version 4]; ", > 2. "status": 500 > > I noticed RemoteException: Server IPC version 7 cannot communicate with > client version 4 > > #2 Then I tried hadoop2 version,. So I installed plugin with this command > bin/plugin --install > elasticsearch/elasticsearch-repository-hdfs/2.0.0-hadoop2 > > I sent a PUT request as above, this time I even got more strange > exectiopon > > NoClassDefFoundError[org/apache/commons/cli/ParseException] > Here is the response. > > { > "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: > > 1. Error injecting constructor, java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException > at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) > while locating org.elasticsearch.repositories.hdfs.HdfsRepository > while locating org.elasticsearch.repositories.Repository > > 1 error]; nested: NoClassDefFoundError[org/apache/commons/cli/ParseException]; nested: ClassNotFoundException[org.apache.commons.cli.ParseException]; ", > "status": 500 > } > > I wonder if any one have simiar experiences. Note the failed cases are actaully > more realistic deplyoment choices. Because my hadoop cluster will less likely be on the same node > as my ES server. > Thanks, > Jack -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/259f35c0-32f1-45fe-8b3c-0e4f69c30ca5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/259f35c0-32f1-45fe-8b3c-0e4f69c30ca5%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mateusz\_Kaczynski](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mateusz_kaczynski/32/1206_2.png) [@Mateusz\_Kaczynski](https://discuss.elastic.co/u/Mateusz_Kaczynski)\ **Post date:** [October 14, 2014, 4:21pm UTC](https://discuss.elastic.co/t/issues-with-using-repository-hdfs-plug-in-for-snapshot-restore-operation/18513/4 "2014-10-14T16:21:14Z")
I had a similar scenario, running CDH 4.6, unable to initialise the repository with hadoop2 version and started changing gradle as Brent suggests. However, maintaining our own version was a bit too much in our case. As Costin pointed to me here [https://groups.google.com/forum/?fromgroups=#!topic/elasticsearch/613YHEUAtuA](https://groups.google.com/forum/?fromgroups=#!topic/elasticsearch/613YHEUAtuA), the lightweight jar is there for the plethora of distributions, as long as you have your hadoop jars on the classpath. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/dffaaab2-8e66-4574-a755-a1577a6343bc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/dffaaab2-8e66-4574-a755-a1577a6343bc%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Daniel\_Gligorov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_gligorov/32/872_2.png) [@Daniel\_Gligorov](https://discuss.elastic.co/u/Daniel_Gligorov)\ **Post date:** [November 21, 2014, 11:00pm UTC](https://discuss.elastic.co/t/issues-with-using-repository-hdfs-plug-in-for-snapshot-restore-operation/18513/5 "2014-11-21T23:00:11Z")
Hey Jinyuan, I'm having the same issue and want to solve with the light version. How are you doing this part `hadoop classpath` to ES\_CLASSPATH? I tried to exec `hadoop class' path and added the result to "export ES\_CLASSPATH=" but getting same error, is that what you ment? Thank you, On Monday, July 7, 2014 2:42:04 PM UTC-7, Jinyuan Zhou wrote: > I am using elasticsearch 1.2.1 and CDH 4.6. quick start vm. My ES server > is installed on the same vm. > I have one successful senario: I used light version and add the result and > command `hadoop classpath` to ES\_CLASSPATH > > But I encoutered errros with the default version and hadoop2 version. > Here is the details of issues. > #1. I installed the plugin with this command > bin/plugin --install elasticsearch/elasticsearch-repository-hdfs/2.0.0 > and I sent a PUT request below: > url: [http://localhost:9200/\_snapshot/hdfs\_repo](http://localhost:9200/_snapshot/hdfs_repo) > data :{ > "type":"hdfs", > "settings": > { > "uri":"hdfs://localhost.localdomain:8020", > "path":"/user/cloudera/es\_snapshot" > } > } > > I got this response > > 1. "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: > > 1. Error injecting constructor, org.elasticsearch.ElasticsearchGenerationException: Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020] > at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) > while locating org.elasticsearch.repositories.hdfs.HdfsRepository > while locating org.elasticsearch.repositories.Repository > > 1 error]; nested: ElasticsearchGenerationException[Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020]]; nested: RemoteException[Server IPC version 7 cannot communicate with client version 4]; ", > 2. "status": 500 > > I noticed RemoteException: Server IPC version 7 cannot communicate with > client version 4 > > #2 Then I tried hadoop2 version,. So I installed plugin with this command > bin/plugin --install > elasticsearch/elasticsearch-repository-hdfs/2.0.0-hadoop2 > > I sent a PUT request as above, this time I even got more strange > exectiopon > > NoClassDefFoundError[org/apache/commons/cli/ParseException] > Here is the response. > > { > "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: > > 1. Error injecting constructor, java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException > at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) > while locating org.elasticsearch.repositories.hdfs.HdfsRepository > while locating org.elasticsearch.repositories.Repository > > 1 error]; nested: NoClassDefFoundError[org/apache/commons/cli/ParseException]; nested: ClassNotFoundException[org.apache.commons.cli.ParseException]; ", > "status": 500 > } > > I wonder if any one have simiar experiences. Note the failed cases are actaully > more realistic deplyoment choices. Because my hadoop cluster will less likely be on the same node > as my ES server. > Thanks, > Jack -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e1f8dbcc-1c2e-46bc-8176-a40a6a5f0692%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e1f8dbcc-1c2e-46bc-8176-a40a6a5f0692%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jinyuan\_Zhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinyuan_zhou/32/90185_2.png) [@Jinyuan\_Zhou](https://discuss.elastic.co/u/Jinyuan_Zhou)\ **Post date:** [November 22, 2014, 12:17am UTC](https://discuss.elastic.co/t/issues-with-using-repository-hdfs-plug-in-for-snapshot-restore-operation/18513/6 "2014-11-22T00:17:28Z")
Yes. the output of "hadoop classpath" should give you a list of jars that a Hadoop client needed to talk to hadoop cluster. as long as you put these in Elasticsearch's classpath, it should eliminate many common issues like wrong version of jar get loaded or a wrong hadoop client version is used to talk to cluster make. Make sure you follow classpath's format when you append output to ES\_CLASSPATH On Friday, November 21, 2014 3:00:11 PM UTC-8, Daniel Gligorov wrote: > Hey Jinyuan, > > I'm having the same issue and want to solve with the light version. > How are you doing this part `hadoop classpath` to ES\_CLASSPATH? > > I tried to exec `hadoop class' path and added the result to "export > ES\_CLASSPATH=" but getting same error, is that what you ment? > > Thank you, > > On Monday, July 7, 2014 2:42:04 PM UTC-7, Jinyuan Zhou wrote: > > > I am using elasticsearch 1.2.1 and CDH 4.6. quick start vm. My ES server > > is installed on the same vm. > > I have one successful senario: I used light version and add the result > > and command `hadoop classpath` to ES\_CLASSPATH > > > > But I encoutered errros with the default version and hadoop2 version. > > Here is the details of issues. > > #1. I installed the plugin with this command > > bin/plugin --install elasticsearch/elasticsearch-repository-hdfs/2.0.0 > > and I sent a PUT request below: > > url: [http://localhost:9200/\_snapshot/hdfs\_repo](http://localhost:9200/_snapshot/hdfs_repo) > > data :{ > > "type":"hdfs", > > "settings": > > { > > "uri":"hdfs://localhost.localdomain:8020", > > "path":"/user/cloudera/es\_snapshot" > > } > > } > > > > I got this response > > > > 1. "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: > > > > 1. Error injecting constructor, org.elasticsearch.ElasticsearchGenerationException: Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020] > > at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) > > while locating org.elasticsearch.repositories.hdfs.HdfsRepository > > while locating org.elasticsearch.repositories.Repository > > > > 1 error]; nested: ElasticsearchGenerationException[Cannot create Hdfs file-system for uri [hdfs://localhost.localdomain:8020]]; nested: RemoteException[Server IPC version 7 cannot communicate with client version 4]; ", > > 2. "status": 500 > > > > I noticed RemoteException: Server IPC version 7 cannot communicate with > > client version 4 > > > > #2 Then I tried hadoop2 version,. So I installed plugin with this command > > bin/plugin --install > > elasticsearch/elasticsearch-repository-hdfs/2.0.0-hadoop2 > > > > I sent a PUT request as above, this time I even got more strange > > exectiopon > > > > NoClassDefFoundError[org/apache/commons/cli/ParseException] > > Here is the response. > > > > { > > "error": "RepositoryException[[hdfs\_repo] failed to create repository]; nested: CreationException[Guice creation errors: > > > > 1. Error injecting constructor, java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException > > at org.elasticsearch.repositories.hdfs.HdfsRepository.(Unknown Source) > > while locating org.elasticsearch.repositories.hdfs.HdfsRepository > > while locating org.elasticsearch.repositories.Repository > > > > 1 error]; nested: NoClassDefFoundError[org/apache/commons/cli/ParseException]; nested: ClassNotFoundException[org.apache.commons.cli.ParseException]; ", > > "status": 500 > > } > > > > I wonder if any one have simiar experiences. Note the failed cases are actaully > > more realistic deplyoment choices. Because my hadoop cluster will less likely be on the same node > > as my ES server. > > Thanks, > > Jack -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e3bafea8-5fcc-4801-8c43-90056c7c05a4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e3bafea8-5fcc-4801-8c43-90056c7c05a4%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:48am UTC](https://discuss.elastic.co/t/issues-with-using-repository-hdfs-plug-in-for-snapshot-restore-operation/18513/7 "2017-07-06T00:48:22Z")