How to find the right plugin to save data in hdfs?


(Julian Zhang) #1

Hi all

I want to set my ES to save all data to hdfs.

I had find the project https://github.com/elastic/elasticsearch-hdfs/ and the new project https://github.com/elastic/elasticsearch-hadoop/tree/master/repository-hdfs

The plugin install command in github is "bin/plugin -i elasticsearch/elasticsearch-repository-hdfs/2.1.0"

I had try to change plugin to "elasticsearch/elasticsearch-repository-hdfs/2.1.0-light","elasticsearch/elasticsearch-repository-hdfs/2.1.0-hadoop2" , "elasticsearch/elasticsearch-repository-hdfs/2.10". But every time the install failed at verify step. The output is as blow.

So I want to know, what is the plugin name I can use to install by the command "plugin isntall"?

My ES version is 2.0beta, hadoop version is 2.6.0(CDH5.4.1).

Verifying http://download.elastic.co/elasticsearch/elasticsearch-repository-hdfs/elasticsearch-repository-hdfs-2.1.0.zip checksums if available ...
Trying http://search.maven.org/remotecontent?filepath=elasticsearch/elasticsearch-repository-hdfs/2.1.0/elasticsearch-repository-hdfs-2.1.0.zip ...
Trying https://oss.sonatype.org/service/local/repositories/releases/content/elasticsearch/elasticsearch-repository-hdfs/2.1.0/elasticsearch-repository-hdfs-2.1.0.zip ...
Trying https://github.com/elasticsearch/elasticsearch-repository-hdfs/archive/2.1.0.zip ...
Trying https://github.com/elasticsearch/elasticsearch-repository-hdfs/archive/master.zip ...
NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)
ERROR: /tmp/elasticsearch-repository-hdfs5019802374743263750.zip

(Mark Walkom) #2

What do you mean by this. Do you want a copy of all data in HDFS as well as ES? Or do you want ES to store its data on HDFS?

Because you cannot do the second one.


(system) #3