Trouble With Elasticsearch Hadoop Integration

Ed_Brown_VSTI · September 13, 2011, 9:40pm

Hello,
I'm having trouble with the integration of Elasticsearch and
Cloudera's version of Hadoop. Elasticsearch fails to start.

Elasticsearch version: 0.17.6
Cloudera Hadoop: 0.20.1

I also installed the Hadoop plugin by using the following:
plugin -install hadoop

The following lines are in the elasticsearch.yaml file:

===

gateway.type: hdfs
gateway.hdfs.uri: hdfs://127.0.0.1:8020
gateway.hdfs.path: /user/hdfs/output_multi

===

I have verified the Hadoop name node installation by creating files in
HDFS and using the name node to list the files. The output is as
follows:

==== Hadoop command output start ====

hadoop fs -fs hdfs://127.0.0.1:8020 -ls /user/hdfs/output_multi
Found 3 items
-rw-r--r-- 1 hdfs supergroup 0 2011-09-13 17:17 /user/hdfs/
output_multi/_SUCCESS
drwxr-xr-x - hdfs supergroup 0 2011-09-13 17:05 /user/hdfs/
output_multi/_logs
-rw-r--r-- 1 hdfs supergroup 129962 2011-09-13 17:14 /user/hdfs/
output_multi/part-r-00000

==== Hadoop command output end ====

When I start Elasticsearch, it fails with the following error.

== Error in Elasticsearch log ==

{elasticsearch/0.17.6}: Initialization Failed ...

IOException[Call to localhost/127.0.0.1:8020 failed on local
exception: java.io.EOFException]

== End Elasticsearch log error ==

At the same time, the following error appears in the Hadoop name node
log file:

== Error in Hadoop name node log ==

2011-09-13 17:20:34,439 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from 127.0.0.1:35627 got version 3 expected
version 4
2011-09-13 17:20:34,566 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from 127.0.0.1:35628 got version 3 expected
version 4

== End Hadoop name node log ==

Any help would be appreciated.

TYIA

Ed_Brown_VSTI · September 13, 2011, 9:44pm

Sorry, it's Coludera Hadoop: 0.20.2

On Sep 13, 5:40 pm, Ed Brown VSTI edbrownv...@gmail.com wrote:

Hello,
I'm having trouble with the integration of Elasticsearch and
Cloudera's version of Hadoop. Elasticsearch fails to start.

Elasticsearch version: 0.17.6
Cloudera Hadoop: 0.20.1

I also installed the Hadoop plugin by using the following:
plugin -install hadoop

The following lines are in the elasticsearch.yaml file:

===

gateway.type: hdfs
gateway.hdfs.uri: hdfs://127.0.0.1:8020
gateway.hdfs.path: /user/hdfs/output_multi

===

I have verified the Hadoop name node installation by creating files in
HDFS and using the name node to list the files. The output is as
follows:

==== Hadoop command output start ====

hadoop fs -fs hdfs://127.0.0.1:8020 -ls /user/hdfs/output_multi
Found 3 items
-rw-r--r-- 1 hdfs supergroup 0 2011-09-13 17:17 /user/hdfs/
output_multi/_SUCCESS
drwxr-xr-x - hdfs supergroup 0 2011-09-13 17:05 /user/hdfs/
output_multi/_logs
-rw-r--r-- 1 hdfs supergroup 129962 2011-09-13 17:14 /user/hdfs/
output_multi/part-r-00000

==== Hadoop command output end ====

When I start Elasticsearch, it fails with the following error.

== Error in Elasticsearch log ==

{elasticsearch/0.17.6}: Initialization Failed ...

IOException[Call to localhost/127.0.0.1:8020 failed on local
exception: java.io.EOFException]

== End Elasticsearch log error ==

At the same time, the following error appears in the Hadoop name node
log file:

== Error in Hadoop name node log ==

2011-09-13 17:20:34,439 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from 127.0.0.1:35627 got version 3 expected
version 4
2011-09-13 17:20:34,566 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from 127.0.0.1:35628 got version 3 expected
version 4

== End Hadoop name node log ==

Any help would be appreciated.

TYIA

philwhln · September 14, 2011, 5:00am

Hi Ed,

On Tue, Sep 13, 2011 at 2:40 PM, Ed Brown VSTI edbrownvsti@gmail.comwrote:

Elasticsearch version: 0.17.6
Cloudera Hadoop: 0.20.1

== Error in Hadoop name node log ==

2011-09-13 17:20:34,439 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from 127.0.0.1:35627 got version 3 expected
version 4

Sounds like you have mismatching versions of Hadoop jars.

Elasticsearch 0.17.6 uses hadoop-core:0.20.2, whereas you're using Apache
Hadoop 0.20.1 (via Cloudera's CDH2)

Install Cloudera's CDH3 and you should be all good.

Cheers,
Phil

http://www.philwhln.com

kimchy · September 14, 2011, 7:48am

Another option is to copy the hadoop jar into the plugin instead of the one
elasticsearch comes with.

On Wed, Sep 14, 2011 at 8:00 AM, Phil Whelan phil123@gmail.com wrote:

Hi Ed,

On Tue, Sep 13, 2011 at 2:40 PM, Ed Brown VSTI edbrownvsti@gmail.comwrote:

Elasticsearch version: 0.17.6
Cloudera Hadoop: 0.20.1

== Error in Hadoop name node log ==

2011-09-13 17:20:34,439 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from 127.0.0.1:35627 got version 3 expected
version 4

Sounds like you have mismatching versions of Hadoop jars.

Elasticsearch 0.17.6 uses hadoop-core:0.20.2, whereas you're using Apache
Hadoop 0.20.1 (via Cloudera's CDH2)

Install Cloudera's CDH3 and you should be all good.

Cheers,
Phil

http://www.philwhln.com

Ed_Brown_VSTI · September 14, 2011, 1:07pm

Phil,
That was EXACTLY the problem!
The Hadoop jar that was in the plugin directory was MUCH SMALLER than
the Hadoop jar provided by Cloudera. Once I switched jars, all was
well!

On Sep 14, 1:00 am, Phil Whelan phil...@gmail.com wrote:

Hi Ed,

On Tue, Sep 13, 2011 at 2:40 PM, Ed Brown VSTI edbrownv...@gmail.comwrote:

Elasticsearch version: 0.17.6
Cloudera Hadoop: 0.20.1

== Error in Hadoop name node log ==

2011-09-13 17:20:34,439 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from 127.0.0.1:35627 got version 3 expected
version 4

Sounds like you have mismatching versions of Hadoop jars.

Elasticsearch 0.17.6 uses hadoop-core:0.20.2, whereas you're using Apache
Hadoop 0.20.1 (via Cloudera's CDH2)

Install Cloudera's CDH3 and you should be all good.

Cheers,
Phil

http://www.philwhln.com

Ed_Brown_VSTI · September 14, 2011, 1:08pm

Yes, I switched the plugin jar and all was well.

I march on!

On Sep 14, 3:48 am, Shay Banon kim...@gmail.com wrote:

Another option is to copy the hadoop jar into the plugin instead of the one
elasticsearch comes with.

On Wed, Sep 14, 2011 at 8:00 AM, Phil Whelan phil...@gmail.com wrote:

Hi Ed,

On Tue, Sep 13, 2011 at 2:40 PM, Ed Brown VSTI edbrownv...@gmail.comwrote:

Elasticsearch version: 0.17.6
Cloudera Hadoop: 0.20.1

== Error in Hadoop name node log ==

2011-09-13 17:20:34,439 WARN org.apache.hadoop.ipc.Server: Incorrect
header or version mismatch from 127.0.0.1:35627 got version 3 expected
version 4

Sounds like you have mismatching versions of Hadoop jars.

Elasticsearch 0.17.6 uses hadoop-core:0.20.2, whereas you're using Apache
Hadoop 0.20.1 (via Cloudera's CDH2)

Install Cloudera's CDH3 and you should be all good.

Cheers,
Phil

http://www.philwhln.com