ES-Hive


(P lva) #1

Hi, I have few weblogs in a hive table that I'd like to visualize in kibana.
ES is on the same node as hive server.

Followed directions from this
page http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

I can create a table using esstorage handler, but when I tried to ingest
data into this table I got

Error: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {first row of my table}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row {
** first row of my table**}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.io.IOException: Out of nodes and retries; caught exception
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)
at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)
at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
... 9 more
Caused by: java.io.IOException: Out of nodes and retries; caught exception
at
org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)
at
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)
at
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)
at
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)
at
org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)
at
org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)
at
org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)
at
org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)
... 18 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.(Socket.java:425)
at java.net.Socket.(Socket.java:280)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
at
org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at
org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)
at
org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)
... 27 more

Now, I changed the config network.host to the ipadress of the server. Now
when I run hive insert data i get
FAILED: IllegalStateException Cannot discover Elasticsearch version

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #2

What does your Hive script look like? Can you confirm the ip/address of your Hive and Elasticsearch ? How are you
executing the script?
The error indicates an error in your network configuration.

Cheers,

P.S. Feel free to post a gist or whatever it's convenient.

On 3/13/2014 10:38 PM, P lva wrote:

Hi, I have few weblogs in a hive table that I'd like to visualize in kibana.
ES is on the same node as hive server.

Followed directions from this page http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

I can create a table using esstorage handler, but when I tried to ingest data into this table I got

Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing
row {first row of my table}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {
** first row of
my table**}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: Out of nodes and retries; caught exception
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
... 9 more
Caused by: java.io.IOException: Out of nodes and retries; caught exception
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)
at org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)
at org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)
at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)
... 18 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.(Socket.java:425)
at java.net.Socket.(Socket.java:280)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)
... 27 more

Now, I changed the config network.host to the ipadress of the server. Now when I run hive insert data i get
FAILED: IllegalStateException Cannot discover Elasticsearch version

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/532222EE.5040803%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(P lva) #3

I have a simple query
insert into table eslogs select * from eslogs_ext;
Hive and elasticsearch are running on the same host.

To execute the script I'm following the directions from the link.
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

There are two elasticsearch nodes, and they can recognize each other (as
indicated by start up process) , but why would hive not be able to pick
them up ? Can you explain what could have gone wrong ?

On Thursday, March 13, 2014 4:28:14 PM UTC-5, Costin Leau wrote:

What does your Hive script look like? Can you confirm the ip/address of
your Hive and Elasticsearch ? How are you
executing the script?
The error indicates an error in your network configuration.

Cheers,

P.S. Feel free to post a gist or whatever it's convenient.

On 3/13/2014 10:38 PM, P lva wrote:

Hi, I have few weblogs in a hive table that I'd like to visualize in
kibana.
ES is on the same node as hive server.

Followed directions from this page
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

I can create a table using esstorage handler, but when I tried to
ingest data into this table I got

Error: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing
row {*first row of my table}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)

     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) 

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row {*** first row of
my table**}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.io.IOException: Out of nodes and retries; caught exception
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)

     at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

     at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

     at 

org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)

     at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

     at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

     at 

org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)

     at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

     at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

     at 

org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)

     ... 9 more 

Caused by: java.io.IOException: Out of nodes and retries; caught
exception
at
org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)
at
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)
at
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)
at
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)
at
org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)
at
org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)
at
org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)

     at 

org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)

     at 

org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)

     at 

org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)

     ... 18 more 

Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

     at 

java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

     at 

java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

     at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) 
     at java.net.Socket.connect(Socket.java:579) 
     at java.net.Socket.connect(Socket.java:528) 
     at java.net.Socket.<init>(Socket.java:425) 
     at java.net.Socket.<init>(Socket.java:280) 
     at 

org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)

     at 

org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)

     at 

org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)

     at 

org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)

     at 

org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)

     at 

org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)

     at 

org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)

     at 

org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)

     at 

org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)

     ... 27 more 

Now, I changed the config network.host to the ipadress of the server.
Now when I run hive insert data i get
FAILED: IllegalStateException Cannot discover Elasticsearch version

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:> <mailto:
elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com

<
https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #4

Without any extra information I'm afraid I can only guess what might be the issue.
Make sure you have the latest Elasticsearch 0.90 or 1.x available on port 9200 with the HTTP/REST port open.
Also make sure that Hive actually runs on the same machine - not just the client but also the server (meaning Hadoop
itself).

You indicate that if you change the network configuration you get an error regarding the version - this suggests that:

  1. Hive is actually running on a different machine than ES - hence the network error
  2. After pointing Hive to the actual ES machine, you get an error since you're using an old Elasticsearch version (0.20)

Cheers,

On 3/14/2014 12:19 AM, P lva wrote:

I have a simple query
insert into table eslogs select * from eslogs_ext;
Hive and elasticsearch are running on the same host.

To execute the script I'm following the directions from the link.
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

There are two elasticsearch nodes, and they can recognize each other (as indicated by start up process) , but why would
hive not be able to pick them up ? Can you explain what could have gone wrong ?

On Thursday, March 13, 2014 4:28:14 PM UTC-5, Costin Leau wrote:

What does your Hive script look like? Can you confirm the ip/address of your Hive and Elasticsearch ? How are you
executing the script?
The error indicates an error in your network configuration.

Cheers,

P.S. Feel free to post a gist or whatever it's convenient.

On 3/13/2014 10:38 PM, P lva wrote:
> Hi, I have few weblogs in a hive table that I'd like to visualize in kibana.
> ES is on the same node as hive server.
>
> Followed directions from this pagehttp://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>
>
> I can create a table  using esstorage handler, but when I tried to ingest data into this table I got
>
> Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing
> row {***first row of my table**}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
>          at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>          at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
>          at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>          at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
>          at java.security.AccessController.doPrivileged(Native Method)
>          at javax.security.auth.Subject.doAs(Subject.java:415)
>          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>          at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {*** first row of
> my table**}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: Out of nodes and retries; caught exception
>          at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)
>          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>          at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)
>          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>          at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
>          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>          at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
>          ... 9 more
> Caused by: java.io.IOException: Out of nodes and retries; caught exception
>          at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)
>          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)
>          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)
>          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)
>          at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)
>          at org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)
>          at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)
>          at org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)
>          at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)
>          at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)
>          ... 18 more
> Caused by: java.net.ConnectException: Connection refused
>          at java.net.PlainSocketImpl.socketConnect(Native Method)
>          at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>          at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>          at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>          at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>          at java.net.Socket.connect(Socket.java:579)
>          at java.net.Socket.connect(Socket.java:528)
>          at java.net.Socket.<init>(Socket.java:425)
>          at java.net.Socket.<init>(Socket.java:280)
>          at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
>          at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
>          at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>          at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>          at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>          at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>          at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
>          at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)
>          at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)
>          ... 27 more
>
> Now, I changed the config network.host to the ipadress of the server. Now when I run hive insert data i get
> FAILED: IllegalStateException Cannot discover Elasticsearch version
>
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
>elasticsearc...@googlegroups.com <javascript:> <mailto:elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
> To view this discussion on the web visit
>https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>
> <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> For more options, visithttps://groups.google.com/d/optout <https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5323213A.7060200%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(P lva) #5

Costin,

About what you asked,

  1. Hive server is running on the same machine as elasticsearch,
  2. I get a response when I do a curl http://elasticsearc&hivehost:9200

I feel I'm missing something simple. This is what I've got until now.

  1. First error (connection refused) is because i left the default settings
    as is.

I changed the 'network.host' in elasticsearch.yml to the hostname

  1. Second error (cannot discover elasticsearch version) is when I changed
    the 'network.host' in elasticsearch.yml to the hostname.

Looked at the source code and figured I'm expected to pass eshost:port to
the hive table at table creation.
(https://github.com/elasticsearch/elasticsearch-hadoop/blob/master/src/main/java/org/elasticsearch/hadoop/hive/EsStorageHandler.java#L48)
.
So I included 'es.resource.write'='http://elasticsearchhost:9200' as one of
the table properties during table creation step.

STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'esdemo/hive',
'es.mapping.name'='time:@timestamp',
'es.resource.write'='http://eshost:9200');

Now I get connection refused again. Is that the right way to pass that
information ?

Thanks for you patience and help

On Friday, March 14, 2014 10:33:14 AM UTC-5, Costin Leau wrote:

Without any extra information I'm afraid I can only guess what might be
the issue.
Make sure you have the latest Elasticsearch 0.90 or 1.x available on port
9200 with the HTTP/REST port open.
Also make sure that Hive actually runs on the same machine - not just the
client but also the server (meaning Hadoop
itself).

You indicate that if you change the network configuration you get an error
regarding the version - this suggests that:

  1. Hive is actually running on a different machine than ES - hence the
    network error
  2. After pointing Hive to the actual ES machine, you get an error since
    you're using an old Elasticsearch version (0.20)

Cheers,

On 3/14/2014 12:19 AM, P lva wrote:

I have a simple query
insert into table eslogs select * from eslogs_ext;
Hive and elasticsearch are running on the same host.

To execute the script I'm following the directions from the link.

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

There are two elasticsearch nodes, and they can recognize each other (as
indicated by start up process) , but why would
hive not be able to pick them up ? Can you explain what could have gone
wrong ?

On Thursday, March 13, 2014 4:28:14 PM UTC-5, Costin Leau wrote:

What does your Hive script look like? Can you confirm the ip/address 

of your Hive and Elasticsearch ? How are you

executing the script? 
The error indicates an error in your network configuration. 

Cheers, 

P.S. Feel free to post a gist or whatever it's convenient. 

On 3/13/2014 10:38 PM, P lva wrote: 
> Hi, I have few weblogs in a hive table that I'd like to visualize 

in kibana.

> ES is on the same node as hive server. 
> 
> Followed directions from this pagehttp://

www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>

> 
> I can create a table  using esstorage handler, but when I tried to 

ingest data into this table I got

> 
> Error: java.lang.RuntimeException: 

org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing

> row {***first row of my table**} 
> at 

org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)

>          at 

org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

>          at 

org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)

>          at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) 
>          at 

org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)

>          at java.security.AccessController.doPrivileged(Native 

Method)

>          at javax.security.auth.Subject.doAs(Subject.java:415) 
>          at 

org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)

>          at 

org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)

> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive 

Runtime Error while processing row {*** first row of

> my table**} 
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 

java.io.IOException: Out of nodes and retries; caught exception

>          at 

org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)

>          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>          at 

org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)

>          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>          at 

org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)

>          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>          at 

org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)

>          ... 9 more 
> Caused by: java.io.IOException: Out of nodes and retries; caught 

exception

>          at 

org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)

>          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)

>          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)

>          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)

>          at 

org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)

>          at 

org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)

>          at 

org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)

>          at 

org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)

>          at 

org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)

>          at 

org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)

>          ... 18 more 
> Caused by: java.net.ConnectException: Connection refused 
>          at java.net.PlainSocketImpl.socketConnect(Native Method) 
>          at 

java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

>          at 

java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

>          at 

java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

>          at 

java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

>          at java.net.Socket.connect(Socket.java:579) 
>          at java.net.Socket.connect(Socket.java:528) 
>          at java.net.Socket.<init>(Socket.java:425) 
>          at java.net.Socket.<init>(Socket.java:280) 
>          at 
> 

org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)

>          at 
> 

org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)

>          at 

org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)

>          at 

org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)

>          at 

org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)

>          at 

org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)

>          at 

org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)

>          at 

org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)

>          at 

org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)

>          ... 27 more 
> 
> Now, I changed the config network.host to the ipadress of the 

server. Now when I run hive insert data i get

> FAILED: IllegalStateException Cannot discover Elasticsearch 

version

> 
> 
> -- 
> You received this message because you are subscribed to the Google 

Groups "elasticsearch" group.

> To unsubscribe from this group and stop receiving emails from it, 

send an email to

>elasticsearc...@googlegroups.com <javascript:> <mailto:

elasticsearch+unsubscribe@googlegroups.com <javascript:> <javascript:>>.

> To view this discussion on the web visit 
>

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>

> <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> For more options, visithttps://groups.google.com/d/optout <

https://groups.google.com/d/optout>.

-- 
Costin 

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:> <mailto:
elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com

<
https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #6

You should not change the configuration in Elasticsearch. By default, ES binds on all available interfaces - specifying
an Ip:port is likely to restrict access rather than extend it.

To find out the configuration options in es-hadoop, look no further than the docs [1]. If the hive job is running on the
same machine as elasticsearch, the default (localhost:9200) should be enough. As an alternative you can specify the
public ip of your host: x.y.z.w

As a last step, enable TRACE logging (through log4j) on org.elasticsearch.hadoop package and see what comes out
including network connectivity.

By the way, make sure you don't have any firewall or proxy set on your system which might be picked up automatically by
the JVM.

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html

On 3/14/2014 6:50 PM, P lva wrote:

Costin,

About what you asked,

  1. Hive server is running on the same machine as elasticsearch,
  2. I get a response when I do a curl http://elasticsearc&hivehost:9200

I feel I'm missing something simple. This is what I've got until now.

  1. First error (connection refused) is because i left the default settings as is.

I changed the 'network.host' in elasticsearch.yml to the hostname

  1. Second error (cannot discover elasticsearch version) is when I changed the 'network.host' in elasticsearch.yml to the
    hostname.

Looked at the source code and figured I'm expected to pass eshost:port to the hive table at table creation.
(https://github.com/elasticsearch/elasticsearch-hadoop/blob/master/src/main/java/org/elasticsearch/hadoop/hive/EsStorageHandler.java#L48)
.
So I included 'es.resource.write'='http://elasticsearchhost:9200' as one of the table properties during table creation step.

STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'esdemo/hive',
'es.mapping.name'='time:@timestamp',
'es.resource.write'='http://eshost:9200');

Now I get connection refused again. Is that the right way to pass that information ?

Thanks for you patience and help

On Friday, March 14, 2014 10:33:14 AM UTC-5, Costin Leau wrote:

Without any extra information I'm afraid I can only guess what might be the issue.
Make sure you have the latest Elasticsearch 0.90 or 1.x available on port 9200 with the HTTP/REST port open.
Also make sure that Hive actually runs on the same machine - not just the client but also the server (meaning Hadoop
itself).

You indicate that if you change the network configuration you get an error regarding the version - this suggests that:

1. Hive is actually running on a different machine than ES - hence the network error
2. After pointing Hive to the actual ES machine, you get an error since you're using an old Elasticsearch version
(0.20)

Cheers,

On 3/14/2014 12:19 AM, P lva wrote:
> I have a simple query
> insert into table eslogs select * from eslogs_ext;
> Hive and elasticsearch are running on the same host.
>
> To execute the script I'm following the directions from the link.
>http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>
>
> There are two elasticsearch nodes, and they can recognize each other (as indicated by start up process) , but why would
> hive not be able to pick them up ? Can you explain what could have gone wrong ?
>
>
> On Thursday, March 13, 2014 4:28:14 PM UTC-5, Costin Leau wrote:
>
>     What does your Hive script look like? Can you confirm the ip/address of your Hive and Elasticsearch ? How are you
>     executing the script?
>     The error indicates an error in your network configuration.
>
>     Cheers,
>
>     P.S. Feel free to post a gist or whatever it's convenient.
>
>     On 3/13/2014 10:38 PM, P lva wrote:
>     > Hi, I have few weblogs in a hive table that I'd like to visualize in kibana.
>     > ES is on the same node as hive server.
>     >
>     > Followed directions from this pagehttp://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>
>     <http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>>
>     >
>     > I can create a table  using esstorage handler, but when I tried to ingest data into this table I got
>     >
>     > Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing
>     > row {***first row of my table**}
>     > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
>     >          at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>     >          at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
>     >          at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     >          at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
>     >          at java.security.AccessController.doPrivileged(Native Method)
>     >          at javax.security.auth.Subject.doAs(Subject.java:415)
>     >          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>     >          at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
>     > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {*** first row of
>     > my table**}
>     > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: Out of nodes and retries; caught exception
>     >          at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)
>     >          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>     >          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>     >          at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)
>     >          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>     >          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>     >          at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
>     >          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>     >          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>     >          at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
>     >          ... 9 more
>     > Caused by: java.io.IOException: Out of nodes and retries; caught exception
>     >          at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)
>     >          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)
>     >          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)
>     >          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)
>     >          at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)
>     >          at org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)
>     >          at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)
>     >          at org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)
>     >          at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)
>     >          at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)
>     >          ... 18 more
>     > Caused by: java.net.ConnectException: Connection refused
>     >          at java.net.PlainSocketImpl.socketConnect(Native Method)
>     >          at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>     >          at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>     >          at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>     >          at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>     >          at java.net.Socket.connect(Socket.java:579)
>     >          at java.net.Socket.connect(Socket.java:528)
>     >          at java.net.Socket.<init>(Socket.java:425)
>     >          at java.net.Socket.<init>(Socket.java:280)
>     >          at
>     > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
>     >          at
>     > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
>     >          at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>     >          at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>     >          at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>     >          at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>     >          at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
>     >          at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)
>     >          at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)
>     >          ... 27 more
>     >
>     > Now, I changed the config network.host to the ipadress of the server. Now when I run hive insert data i get
>     > FAILED: IllegalStateException Cannot discover Elasticsearch version
>     >
>     >
>     > --
>     > You received this message because you are subscribed to the Google Groups "elasticsearch" group.
>     > To unsubscribe from this group and stop receiving emails from it, send an email to
>     >elasticsearc...@googlegroups.com <javascript:> <mailto:elasticsearch+unsubscribe@googlegroups.com <javascript:>
<javascript:>>.
>     > To view this discussion on the web visit
>     >https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>
>     <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>>
>     > <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>

>     <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>>.

>
>     > For more options, visithttps://groups.google.com/d/optout <http://groups.google.com/d/optout> <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>.
>
>     --
>     Costin
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
>elasticsearc...@googlegroups.com <javascript:> <mailto:elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
> To view this discussion on the web visit
>https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com>
> <https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> For more options, visithttps://groups.google.com/d/optout <https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53234670.5040201%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(P lva) #7

I got it to work by replacing
'es.resource.write'='http://eshost:9200'
'es.resource.node'='http://eshost:9200'

as one of the tbl properties in hive.

Thanks for your help Costin.

On Friday, March 14, 2014 1:12:00 PM UTC-5, Costin Leau wrote:

You should not change the configuration in Elasticsearch. By default, ES
binds on all available interfaces - specifying
an Ip:port is likely to restrict access rather than extend it.

To find out the configuration options in es-hadoop, look no further than
the docs [1]. If the hive job is running on the
same machine as elasticsearch, the default (localhost:9200) should be
enough. As an alternative you can specify the
public ip of your host: x.y.z.w

As a last step, enable TRACE logging (through log4j) on
org.elasticsearch.hadoop package and see what comes out
including network connectivity.

By the way, make sure you don't have any firewall or proxy set on your
system which might be picked up automatically by
the JVM.

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html

On 3/14/2014 6:50 PM, P lva wrote:

Costin,

About what you asked,

  1. Hive server is running on the same machine as elasticsearch,
  2. I get a response when I do a curl http://elasticsearc&hivehost:9200

I feel I'm missing something simple. This is what I've got until now.

  1. First error (connection refused) is because i left the default
    settings as is.

I changed the 'network.host' in elasticsearch.yml to the hostname

  1. Second error (cannot discover elasticsearch version) is when I
    changed the 'network.host' in elasticsearch.yml to the
    hostname.

Looked at the source code and figured I'm expected to pass eshost:port
to the hive table at table creation.
(
https://github.com/elasticsearch/elasticsearch-hadoop/blob/master/src/main/java/org/elasticsearch/hadoop/hive/EsStorageHandler.java#L48)

.
So I included 'es.resource.write'='http://elasticsearchhost:9200' as
one of the table properties during table creation step.

STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'esdemo/hive',
'es.mapping.name'='time:@timestamp',
'es.resource.write'='http://eshost:9200');

Now I get connection refused again. Is that the right way to pass that
information ?

Thanks for you patience and help

On Friday, March 14, 2014 10:33:14 AM UTC-5, Costin Leau wrote:

Without any extra information I'm afraid I can only guess what might 

be the issue.

Make sure you have the latest Elasticsearch 0.90 or 1.x available on 

port 9200 with the HTTP/REST port open.

Also make sure that Hive actually runs on the same machine - not 

just the client but also the server (meaning Hadoop

itself). 

You indicate that if you change the network configuration you get an 

error regarding the version - this suggests that:

1. Hive is actually running on a different machine than ES - hence 

the network error

2. After pointing Hive to the actual ES machine, you get an error 

since you're using an old Elasticsearch version

(0.20) 

Cheers, 

On 3/14/2014 12:19 AM, P lva wrote: 
> I have a simple query 
> insert into table eslogs select * from eslogs_ext; 
> Hive and elasticsearch are running on the same host. 
> 
> To execute the script I'm following the directions from the link. 
>

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>

> 
> There are two elasticsearch nodes, and they can recognize each 

other (as indicated by start up process) , but why would

> hive not be able to pick them up ? Can you explain what could have 

gone wrong ?

> 
> 
> On Thursday, March 13, 2014 4:28:14 PM UTC-5, Costin Leau wrote: 
> 
>     What does your Hive script look like? Can you confirm the 

ip/address of your Hive and Elasticsearch ? How are you

>     executing the script? 
>     The error indicates an error in your network configuration. 
> 
>     Cheers, 
> 
>     P.S. Feel free to post a gist or whatever it's convenient. 
> 
>     On 3/13/2014 10:38 PM, P lva wrote: 
>     > Hi, I have few weblogs in a hive table that I'd like to 

visualize in kibana.

>     > ES is on the same node as hive server. 
>     > 
>     > Followed directions from this pagehttp://

www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>

>     <

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>>

>     > 
>     > I can create a table  using esstorage handler, but when I 

tried to ingest data into this table I got

>     > 
>     > Error: java.lang.RuntimeException: 

org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing

>     > row {***first row of my table**} 
>     > at 

org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)

>     >          at 

org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

>     >          at 

org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)

>     >          at 

org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)

>     >          at 

org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)

>     >          at 

java.security.AccessController.doPrivileged(Native Method)

>     >          at 

javax.security.auth.Subject.doAs(Subject.java:415)

>     >          at 

org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)

>     >          at 

org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)

>     > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 

Hive Runtime Error while processing row {*** first row of

>     > my table**} 
>     > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 

java.io.IOException: Out of nodes and retries; caught exception

>     >          at 

org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)

>     >          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>     >          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>     >          at 

org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)

>     >          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>     >          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>     >          at 

org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)

>     >          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>     >          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>     >          at 

org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)

>     >          ... 9 more 
>     > Caused by: java.io.IOException: Out of nodes and retries; 

caught exception

>     >          at 

org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)

>     >          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)

>     >          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)

>     >          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)

>     >          at 

org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)

>     >          at 

org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)

>     >          at 

org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)

>     >          at 

org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)

>     >          at 

org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)

>     >          at 

org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)

>     >          ... 18 more 
>     > Caused by: java.net.ConnectException: Connection refused 
>     >          at java.net.PlainSocketImpl.socketConnect(Native 

Method)

>     >          at 

java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

>     >          at 

java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

>     >          at 

java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

>     >          at 

java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

>     >          at java.net.Socket.connect(Socket.java:579) 
>     >          at java.net.Socket.connect(Socket.java:528) 
>     >          at java.net.Socket.<init>(Socket.java:425) 
>     >          at java.net.Socket.<init>(Socket.java:280) 
>     >          at 
>     > 

org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)

>     >          at 
>     > 

org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)

>     >          at 

org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)

>     >          at 

org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)

>     >          at 

org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)

>     >          at 

org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)

>     >          at 

org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)

>     >          at 

org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)

>     >          at 

org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)

>     >          ... 27 more 
>     > 
>     > Now, I changed the config network.host to the ipadress of 

the server. Now when I run hive insert data i get

>     > FAILED: IllegalStateException Cannot discover Elasticsearch 

version

>     > 
>     > 
>     > -- 
>     > You received this message because you are subscribed to the 

Google Groups "elasticsearch" group.

>     > To unsubscribe from this group and stop receiving emails 

from it, send an email to

>     >elasticsearc...@googlegroups.com <javascript:> <mailto:

elasticsearch+unsubscribe@googlegroups.com <javascript:> <javascript:>

<javascript:>>. 
>     > To view this discussion on the web visit 
>     >

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>

>     <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>>

>     > <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>

>     <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>>.

> 
>     > For more options, visithttps://groups.google.com/d/optout <

http://groups.google.com/d/optout> <https://groups.google.com/d/optout

<https://groups.google.com/d/optout>>. 
> 
>     -- 
>     Costin 
> 
> -- 
> You received this message because you are subscribed to the Google 

Groups "elasticsearch" group.

> To unsubscribe from this group and stop receiving emails from it, 

send an email to

>elasticsearc...@googlegroups.com <javascript:> <mailto:

elasticsearch+unsubscribe@googlegroups.com <javascript:> <javascript:>>.

> To view this discussion on the web visit 
>

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com>

> <

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> For more options, visithttps://groups.google.com/d/optout <

https://groups.google.com/d/optout>.

-- 
Costin 

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:> <mailto:
elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com

<
https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9041befe-dad6-447f-b666-c3384aa32259%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Costin Leau) #8

There is no such property 'es.resource.node' used in Es-Hadoop.
And es.resource.write is something set internally by the Hive support not to mention it should point to a index/type NOT
host:port.

You most likely want:
es.nodes=eshost:9200
and
es.resource=[index]/[type]

See the previous link that I've listed on the properties supported:
[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html

On 3/15/2014 1:48 AM, P lva wrote:

I got it to work by replacing
'es.resource.write'='http://eshost:9200'
'es.resource.node'='http://eshost:9200'

as one of the tbl properties in hive.

Thanks for your help Costin.

On Friday, March 14, 2014 1:12:00 PM UTC-5, Costin Leau wrote:

You should not change the configuration in Elasticsearch. By default, ES binds on all available interfaces - specifying
an Ip:port is likely to restrict access rather than extend it.

To find out the configuration options in es-hadoop, look no further than the docs [1]. If the hive job is running on
the
same machine as elasticsearch, the default (localhost:9200) should be enough. As an alternative you can specify the
public ip of your host: x.y.z.w

As a last step, enable TRACE logging (through log4j) on org.elasticsearch.hadoop package and see what comes out
including network connectivity.

By the way, make sure you don't have any firewall or proxy set on your system which might be picked up automatically by
the JVM.

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html>

On 3/14/2014 6:50 PM, P lva wrote:
> Costin,
>
> About what you asked,
> 1) Hive server is running on the same machine as elasticsearch,
> 2) I get a response when I do a curlhttp://elasticsearc&hivehost:9200
>
>
> I feel I'm missing something simple. This is what I've got until now.
>
> 1) First error (connection refused) is because i left the default settings as is.
>
> I changed the 'network.host' in elasticsearch.yml to the hostname
>
> 2) Second error (cannot discover elasticsearch version) is when I changed the 'network.host' in elasticsearch.yml to the
> hostname.
>
> Looked at the source code and figured I'm expected to pass eshost:port to the hive table at table creation.
> (https://github.com/elasticsearch/elasticsearch-hadoop/blob/master/src/main/java/org/elasticsearch/hadoop/hive/EsStorageHandler.java#L48
<https://github.com/elasticsearch/elasticsearch-hadoop/blob/master/src/main/java/org/elasticsearch/hadoop/hive/EsStorageHandler.java#L48>)

> .
> So I included 'es.resource.write'='http://elasticsearchhost:9200 <http://elasticsearchhost:9200>' as one of the table properties during table creation
step.
>
> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' = 'esdemo/hive',
>                'es.mapping.name <http://es.mapping.name>'='time:@timestamp',
>                'es.resource.write'='http://eshost:9200');
>
> Now I get connection refused again. Is that the right way to pass that information ?
>
> Thanks for you patience and help
>
>
>
>
> On Friday, March 14, 2014 10:33:14 AM UTC-5, Costin Leau wrote:
>
>     Without any extra information I'm afraid I can only guess what might be the issue.
>     Make sure you have the latest Elasticsearch 0.90 or 1.x available on port 9200 with the HTTP/REST port open.
>     Also make sure that Hive actually runs on the same machine - not just the client but also the server (meaning Hadoop
>     itself).
>
>     You indicate that if you change the network configuration you get an error regarding the version - this suggests that:
>
>     1. Hive is actually running on a different machine than ES - hence the network error
>     2. After pointing Hive to the actual ES machine, you get an error since you're using an old Elasticsearch version
>     (0.20)
>
>     Cheers,
>
>     On 3/14/2014 12:19 AM, P lva wrote:
>     > I have a simple query
>     > insert into table eslogs select * from eslogs_ext;
>     > Hive and elasticsearch are running on the same host.
>     >
>     > To execute the script I'm following the directions from the link.
>     >http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>
>     <http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>>
>     >
>     > There are two elasticsearch nodes, and they can recognize each other (as indicated by start up process) , but why would
>     > hive not be able to pick them up ? Can you explain what could have gone wrong ?
>     >
>     >
>     > On Thursday, March 13, 2014 4:28:14 PM UTC-5, Costin Leau wrote:
>     >
>     >     What does your Hive script look like? Can you confirm the ip/address of your Hive and Elasticsearch ? How are you
>     >     executing the script?
>     >     The error indicates an error in your network configuration.
>     >
>     >     Cheers,
>     >
>     >     P.S. Feel free to post a gist or whatever it's convenient.
>     >
>     >     On 3/13/2014 10:38 PM, P lva wrote:
>     >     > Hi, I have few weblogs in a hive table that I'd like to visualize in kibana.
>     >     > ES is on the same node as hive server.
>     >     >
>     >     > Followed directions from this pagehttp://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>
>     <http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>>
>     >     <http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>
>     <http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>>>
>     >     >
>     >     > I can create a table  using esstorage handler, but when I tried to ingest data into this table I got
>     >     >
>     >     > Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing
>     >     > row {***first row of my table**}
>     >     > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
>     >     >          at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>     >     >          at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
>     >     >          at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     >     >          at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
>     >     >          at java.security.AccessController.doPrivileged(Native Method)
>     >     >          at javax.security.auth.Subject.doAs(Subject.java:415)
>     >     >          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>     >     >          at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
>     >     > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {*** first row of
>     >     > my table**}
>     >     > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: Out of nodes and retries; caught exception
>     >     >          at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)
>     >     >          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>     >     >          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>     >     >          at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)
>     >     >          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>     >     >          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>     >     >          at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
>     >     >          at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
>     >     >          at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
>     >     >          at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
>     >     >          ... 9 more
>     >     > Caused by: java.io.IOException: Out of nodes and retries; caught exception
>     >     >          at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)
>     >     >          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)
>     >     >          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)
>     >     >          at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)
>     >     >          at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)
>     >     >          at org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)
>     >     >          at org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)
>     >     >          at org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)
>     >     >          at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)
>     >     >          at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)
>     >     >          ... 18 more
>     >     > Caused by: java.net.ConnectException: Connection refused
>     >     >          at java.net.PlainSocketImpl.socketConnect(Native Method)
>     >     >          at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>     >     >          at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>     >     >          at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>     >     >          at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>     >     >          at java.net.Socket.connect(Socket.java:579)
>     >     >          at java.net.Socket.connect(Socket.java:528)
>     >     >          at java.net.Socket.<init>(Socket.java:425)
>     >     >          at java.net.Socket.<init>(Socket.java:280)
>     >     >          at
>     >     > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
>     >     >          at
>     >     > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
>     >     >          at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>     >     >          at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>     >     >          at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>     >     >          at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>     >     >          at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
>     >     >          at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)
>     >     >          at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)
>     >     >          ... 27 more
>     >     >
>     >     > Now, I changed the config network.host to the ipadress of the server. Now when I run hive insert data i get
>     >     > FAILED: IllegalStateException Cannot discover Elasticsearch version
>     >     >
>     >     >
>     >     > --
>     >     > You received this message because you are subscribed to the Google Groups "elasticsearch" group.
>     >     > To unsubscribe from this group and stop receiving emails from it, send an email to
>     >     >elasticsearc...@googlegroups.com <javascript:> <mailto:elasticsearch+unsubscribe@googlegroups.com <javascript:>
<javascript:>
>     <javascript:>>.
>     >     > To view this discussion on the web visit
>     >     >https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>
>     <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>>
>     >     <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>
>     <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>>>
>     >     > <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>

>     <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>

>
>     >     <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>

>     <https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>>>.

>
>     >
>     >     > For more options, visithttps://groups.google.com/d/optout <http://groups.google.com/d/optout> <http://groups.google.com/d/optout
<http://groups.google.com/d/optout>> <https://groups.google.com/d/optout <https://groups.google.com/d/optout>
>     <https://groups.google.com/d/optout <https://groups.google.com/d/optout>>>.
>     >
>     >     --
>     >     Costin
>     >
>     > --
>     > You received this message because you are subscribed to the Google Groups "elasticsearch" group.
>     > To unsubscribe from this group and stop receiving emails from it, send an email to
>     >elasticsearc...@googlegroups.com <javascript:> <mailto:elasticsearch+unsubscribe@googlegroups.com <javascript:>
<javascript:>>.
>     > To view this discussion on the web visit
>     >https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com>
>     <https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com>>
>     > <https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer>

>     <https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer>>>.

>
>     > For more options, visithttps://groups.google.com/d/optout <http://groups.google.com/d/optout> <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>.
>
>     --
>     Costin
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
>elasticsearc...@googlegroups.com <javascript:> <mailto:elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
> To view this discussion on the web visit
>https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com>
> <https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> For more options, visithttps://groups.google.com/d/optout <https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9041befe-dad6-447f-b666-c3384aa32259%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9041befe-dad6-447f-b666-c3384aa32259%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53240E6B.5020709%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


(P lva) #9

My mistake. It is es.nodes not es.resource.nodes

Thanks again for the help

On Saturday, March 15, 2014 3:25:15 AM UTC-5, Costin Leau wrote:

There is no such property 'es.resource.node' used in Es-Hadoop.
And es.resource.write is something set internally by the Hive support not
to mention it should point to a index/type NOT
host:port.

You most likely want:
es.nodes=eshost:9200
and
es.resource=[index]/[type]

See the previous link that I've listed on the properties supported:
[1]
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html

On 3/15/2014 1:48 AM, P lva wrote:

I got it to work by replacing
'es.resource.write'='http://eshost:9200'
'es.resource.node'='http://eshost:9200'

as one of the tbl properties in hive.

Thanks for your help Costin.

On Friday, March 14, 2014 1:12:00 PM UTC-5, Costin Leau wrote:

You should not change the configuration in Elasticsearch. By 

default, ES binds on all available interfaces - specifying

an Ip:port is likely to restrict access rather than extend it. 

To find out the configuration options in es-hadoop, look no further 

than the docs [1]. If the hive job is running on

the 
same machine as elasticsearch, the default (localhost:9200) should 

be enough. As an alternative you can specify the

public ip of your host: x.y.z.w 

As a last step, enable TRACE logging (through log4j) on 

org.elasticsearch.hadoop package and see what comes out

including network connectivity. 

By the way, make sure you don't have any firewall or proxy set on 

your system which might be picked up automatically by

the JVM. 

[1] 

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/configuration.html>

On 3/14/2014 6:50 PM, P lva wrote: 
> Costin, 
> 
> About what you asked, 
> 1) Hive server is running on the same machine as elasticsearch, 
> 2) I get a response when I do a 

curlhttp://elasticsearc&hivehost:9200

> 
> 
> I feel I'm missing something simple. This is what I've got until 

now.

> 
> 1) First error (connection refused) is because i left the default 

settings as is.

> 
> I changed the 'network.host' in elasticsearch.yml to the hostname 
> 
> 2) Second error (cannot discover elasticsearch version) is when I 

changed the 'network.host' in elasticsearch.yml to the

> hostname. 
> 
> Looked at the source code and figured I'm expected to pass 

eshost:port to the hive table at table creation.

> (

https://github.com/elasticsearch/elasticsearch-hadoop/blob/master/src/main/java/org/elasticsearch/hadoop/hive/EsStorageHandler.java#L48

<

https://github.com/elasticsearch/elasticsearch-hadoop/blob/master/src/main/java/org/elasticsearch/hadoop/hive/EsStorageHandler.java#L48>)

> . 
> So I included 'es.resource.write'='http://elasticsearchhost:9200 <

http://elasticsearchhost:9200>' as one of the table properties during
table creation

step. 
> 
> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' 
> TBLPROPERTIES('es.resource' = 'esdemo/hive', 
>                'es.mapping.name <http://es.mapping.name>'='time:@timestamp', 
>                'es.resource.write'='http://eshost:9200'); 
> 
> Now I get connection refused again. Is that the right way to pass 

that information ?

> 
> Thanks for you patience and help 
> 
> 
> 
> 
> On Friday, March 14, 2014 10:33:14 AM UTC-5, Costin Leau wrote: 
> 
>     Without any extra information I'm afraid I can only guess what 

might be the issue.

>     Make sure you have the latest Elasticsearch 0.90 or 1.x 

available on port 9200 with the HTTP/REST port open.

>     Also make sure that Hive actually runs on the same machine - 

not just the client but also the server (meaning Hadoop

>     itself). 
> 
>     You indicate that if you change the network configuration you 

get an error regarding the version - this suggests that:

> 
>     1. Hive is actually running on a different machine than ES - 

hence the network error

>     2. After pointing Hive to the actual ES machine, you get an 

error since you're using an old Elasticsearch version

>     (0.20) 
> 
>     Cheers, 
> 
>     On 3/14/2014 12:19 AM, P lva wrote: 
>     > I have a simple query 
>     > insert into table eslogs select * from eslogs_ext; 
>     > Hive and elasticsearch are running on the same host. 
>     > 
>     > To execute the script I'm following the directions from the 

link.

>     >

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>

>     <

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>>

>     > 
>     > There are two elasticsearch nodes, and they can recognize 

each other (as indicated by start up process) , but why would

>     > hive not be able to pick them up ? Can you explain what 

could have gone wrong ?

>     > 
>     > 
>     > On Thursday, March 13, 2014 4:28:14 PM UTC-5, Costin Leau 

wrote:

>     > 
>     >     What does your Hive script look like? Can you confirm 

the ip/address of your Hive and Elasticsearch ? How are you

>     >     executing the script? 
>     >     The error indicates an error in your network 

configuration.

>     > 
>     >     Cheers, 
>     > 
>     >     P.S. Feel free to post a gist or whatever it's 

convenient.

>     > 
>     >     On 3/13/2014 10:38 PM, P lva wrote: 
>     >     > Hi, I have few weblogs in a hive table that I'd like 

to visualize in kibana.

>     >     > ES is on the same node as hive server. 
>     >     > 
>     >     > Followed directions from this pagehttp://

www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>

>     <

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>>

>     >     <

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>

>     <

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html

<

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html>>>

>     >     > 
>     >     > I can create a table  using esstorage handler, but 

when I tried to ingest data into this table I got

>     >     > 
>     >     > Error: java.lang.RuntimeException: 

org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing

>     >     > row {***first row of my table**} 
>     >     > at 

org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)

>     >     >          at 

org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

>     >     >          at 

org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)

>     >     >          at 

org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)

>     >     >          at 

org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)

>     >     >          at 

java.security.AccessController.doPrivileged(Native Method)

>     >     >          at 

javax.security.auth.Subject.doAs(Subject.java:415)

>     >     >          at 

org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)

>     >     >          at 

org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)

>     >     > Caused by: 

org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {*** first row of

>     >     > my table**} 
>     >     > Caused by: 

org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: Out
of nodes and retries; caught exception

>     >     >          at 

org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:652)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:88)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)

>     >     >          ... 9 more 
>     >     > Caused by: java.io.IOException: Out of nodes and 

retries; caught exception

>     >     >          at 

org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:81)

>     >     >          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:221)

>     >     >          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:205)

>     >     >          at 

org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:209)

>     >     >          at 

org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:103)

>     >     >          at 

org.elasticsearch.hadoop.rest.RestClient.discoverNodes(RestClient.java:85)

>     >     >          at 

org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:60)

>     >     >          at 

org.elasticsearch.hadoop.mr.EsOutputFormat$ESRecordWriter.init(EsOutputFormat.java:165)

>     >     >          at 

org.elasticsearch.hadoop.hive.EsHiveOutputFormat$ESHiveRecordWriter.write(EsHiveOutputFormat.java:50)

>     >     >          at 

org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:638)

>     >     >          ... 18 more 
>     >     > Caused by: java.net.ConnectException: Connection 

refused

>     >     >          at 

java.net.PlainSocketImpl.socketConnect(Native Method)

>     >     >          at 

java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

>     >     >          at 

java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

>     >     >          at 

java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

>     >     >          at 

java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

>     >     >          at java.net.Socket.connect(Socket.java:579) 
>     >     >          at java.net.Socket.connect(Socket.java:528) 
>     >     >          at java.net.Socket.<init>(Socket.java:425) 
>     >     >          at java.net.Socket.<init>(Socket.java:280) 
>     >     >          at 
>     >     > 

org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)

>     >     >          at 
>     >     > 

org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)

>     >     >          at 

org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)

>     >     >          at 

org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)

>     >     >          at 

org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)

>     >     >          at 

org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)

>     >     >          at 

org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)

>     >     >          at 

org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:160)

>     >     >          at 

org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:74)

>     >     >          ... 27 more 
>     >     > 
>     >     > Now, I changed the config network.host to the ipadress 

of the server. Now when I run hive insert data i get

>     >     > FAILED: IllegalStateException Cannot discover 

Elasticsearch version

>     >     > 
>     >     > 
>     >     > -- 
>     >     > You received this message because you are subscribed 

to the Google Groups "elasticsearch" group.

>     >     > To unsubscribe from this group and stop receiving 

emails from it, send an email to

>     >     >elasticsearc...@googlegroups.com <javascript:> <mailto:

elasticsearch+unsubscribe@googlegroups.com <javascript:> <javascript:>

<javascript:> 
>     <javascript:>>. 
>     >     > To view this discussion on the web visit 
>     >     >

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>

>     <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>>

>     >     <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>

>     <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com>>>

>     >     > <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>

>     <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>

> 
>     >     <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>

>     <

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/a9878590-71b9-46e5-8bd7-7ee3878f959e%40googlegroups.com?utm_medium=email&utm_source=footer>>>>.

> 
>     > 
>     >     > For more options, visithttps://

groups.google.com/d/optout http://groups.google.com/d/optout <
http://groups.google.com/d/optout

<http://groups.google.com/d/optout>> <

https://groups.google.com/d/optout https://groups.google.com/d/optout

>     <https://groups.google.com/d/optout <

https://groups.google.com/d/optout>>>.

>     > 
>     >     -- 
>     >     Costin 
>     > 
>     > -- 
>     > You received this message because you are subscribed to the 

Google Groups "elasticsearch" group.

>     > To unsubscribe from this group and stop receiving emails 

from it, send an email to

>     >elasticsearc...@googlegroups.com <javascript:> <mailto:

elasticsearch+unsubscribe@googlegroups.com <javascript:> <javascript:>

<javascript:>>. 
>     > To view this discussion on the web visit 
>     >

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com>

>     <

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com>>

>     > <

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer>

>     <

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/97ef1333-eb9d-43d7-a3fe-b5e48d072643%40googlegroups.com?utm_medium=email&utm_source=footer>>>.

> 
>     > For more options, visithttps://groups.google.com/d/optout <

http://groups.google.com/d/optout> <https://groups.google.com/d/optout

<https://groups.google.com/d/optout>>. 
> 
>     -- 
>     Costin 
> 
> -- 
> You received this message because you are subscribed to the Google 

Groups "elasticsearch" group.

> To unsubscribe from this group and stop receiving emails from it, 

send an email to

>elasticsearc...@googlegroups.com <javascript:> <mailto:

elasticsearch+unsubscribe@googlegroups.com <javascript:> <javascript:>>.

> To view this discussion on the web visit 
>

https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com>

> <

https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/d1aa80dc-d45b-4372-b287-f7a0c3e8bb46%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> For more options, visithttps://groups.google.com/d/optout <

https://groups.google.com/d/optout>.

-- 
Costin 

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:> <mailto:
elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/9041befe-dad6-447f-b666-c3384aa32259%40googlegroups.com

<
https://groups.google.com/d/msgid/elasticsearch/9041befe-dad6-447f-b666-c3384aa32259%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/95a8c4a6-dcfe-483a-b597-937fe4f8b166%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #10