HDFS setup issues

Elastic Stack Elasticsearch
1

Installed hadoop plugin. But I now see that elastic search is not bound to
port 9200. Any idea why? Before I added gateway commands, all was working
fine, before I changed gateway from local to hdfs.

Here is the conf I added.

===== elasticsearch.yml

gateway.type: hdfs
gateway.hdfs.uri: hdfs://127.0.0.1:54311
gateway.hdfs.path: /app/hadoop/tmp

==========

NOTE: mapred-site is pointing to 54311. Also all elasticsearch/hadoop all
running on a single m/c

====netstat o/p

praveen@praveen-laptop:~/CodeJul/milkyway/sol/server/scripts/api$ sudo
netstat -tapun | grep LISTEN
tcp 0 0 0.0.0.0:35619 0.0.0.0:* LISTEN
4023/java
tcp 0 0 127.0.0.1:54310 0.0.0.0:* LISTEN
4023/java
tcp 0 0 127.0.0.1:54311 0.0.0.0:* LISTEN
4197/java
tcp 0 0 0.0.0.0:4743 0.0.0.0:* LISTEN
1094/openhpid
tcp 0 0 0.0.0.0:59080 0.0.0.0:* LISTEN
4700/java
tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN
4700/java
tcp 0 0 127.0.0.1:34510 0.0.0.0:* LISTEN
4930/java
tcp 0 0 0.0.0.0:50030 0.0.0.0:* LISTEN
4197/java
tcp 0 0 0.0.0.0:28017 0.0.0.0:* LISTEN
2328/mongod
tcp 0 0 127.0.0.1:61619 0.0.0.0:* LISTEN
1224/python2.6
tcp 0 0 0.0.0.0:61620 0.0.0.0:* LISTEN
1224/python2.6
tcp 0 0 0.0.0.0:40598 0.0.0.0:* LISTEN
4197/java
tcp 0 0 0.0.0.0:50070 0.0.0.0:* LISTEN
4023/java

DONOT see 9200 atall (or any port around it)

=====

$ ps -eaf | grep elast
root 10003 1 1 15:14 pts/2 00:00:08 /usr/bin/java -Xms256m
-Xmx256m -Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError -Delasticsearch
-Des.path.home=/home/praveen/es/elasticsearch-0.19.4 -cp
:/home/praveen/es/elasticsearch-0.19.4/lib/:/home/praveen/es/elasticsearch-0.19.4/lib/sigar/
-Des.max-open-files=true -Des.index.storage.type=memory
org.elasticsearch.bootstrap.ElasticSearch

Any help is greatly appreciated !

Thanks
-pk

--

2

Can somebody share their working sample yml file/ map-site xml file ? That
might give me some hints.

Thanks in Advance
-pk

--

3

It is working now.

fs.default.name was pointing to 54311 and not 54310.

The port number in the gateway.hdfs.uri should point to namenode port
(defined in the core-site.xml). My bad.

fs.default.name hdfs://localhost:54310

On Friday, August 10, 2012 5:24:08 PM UTC-7, Praveen Kariyanahalli wrote:

Can somebody share their working sample yml file/ map-site xml file ? That
might give me some hints.

Thanks in Advance
-pk

--

4

It is working now!

The port in the gateway.hdfs.uri was wrongly pointing to 54311 instead of
54310.

Actually it should point to filesystem port number. which is defined in the
core-site.xml

fs.default.name
hdfs://localhost:54310

Regards
-pk

On Friday, August 10, 2012 5:24:08 PM UTC-7, Praveen Kariyanahalli wrote:

Can somebody share their working sample yml file/ map-site xml file ? That
might give me some hints.

Thanks in Advance
-pk

--