Hadoop to ES problem


(philg) #1

hello my context:
ES 0.90.3, elasticsearch-hadoop-1.3.0.M1 , eclipse- java
i am experimenting ES->Hadoop and Hadoop->ES
no problem with Hadoop2ES with these 3 settings ( i see the index + data
through head plugin, perfect)
job.set("es.resource", "eshindex2/eshlog2");
job.set("es.host", "localhost");
job.set("es.port", "9200");
but experimenting the inverse , ie ES 2 Hadoop
....
configuration.set("es.query", "eshindex2/eshlog2/_search?q=*");
configuration.set("es.host", "localhost");
configuration.set("es.port", "9200");
System.out.println("conf:"+ configuration);
JobConf job = new JobConf(configuration,
SimpleJobES2Hadoop.class);
...................
i got the error
10:58:27 INFO mapred.JobClient: Cleaning up the staging area
file:/tmp/hadoop-xxx/mapred/staging/xxx-1302405475/.staging/job_local_0001
Exception in thread "main" java.lang.NullPointerException
at org.elasticsearch.hadoop.rest.dto.Node.(Node.java:30)
at
org.elasticsearch.hadoop.rest.RestClient.getNodes(RestClient.java:139)

Any idea?
bestsregards
philippe

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Costin Leau) #2

If you are using M1 you should use es.resource instead of es.query in your latest example.
es.query is supported in the upcoming 1.3.0.M2 (not in m1).

On 04/02/2014 12:01 PM, Phil gib wrote:

hello my context:
ES 0.90.3, elasticsearch-hadoop-1.3.0.M1 , eclipse- java
i am experimenting ES->Hadoop and Hadoop->ES
no problem with Hadoop2ES with these 3 settings ( i see the index + data through head plugin, perfect)
job.set("es.resource", "eshindex2/eshlog2");
job.set("es.host", "localhost");
job.set("es.port", "9200");
but experimenting the inverse , ie ES 2 Hadoop
....
configuration.set("es.query", "eshindex2/eshlog2/_search?q=*");
configuration.set("es.host", "localhost");
configuration.set("es.port", "9200");
System.out.println("conf:"+ configuration);
JobConf job = new JobConf(configuration, SimpleJobES2Hadoop.class);
...................
i got the error
10:58:27 INFO mapred.JobClient: Cleaning up the staging area
file:/tmp/hadoop-xxx/mapred/staging/xxx-1302405475/.staging/job_local_0001
Exception in thread "main" java.lang.NullPointerException
at org.elasticsearch.hadoop.rest.dto.Node.(Node.java:30)
at org.elasticsearch.hadoop.rest.RestClient.getNodes(RestClient.java:139)

Any idea?
bestsregards
philippe

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/52F0BBE9.5040406%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(philg) #3

Hello Costin,
changing to "es.resource" does not help... :frowning:
argh.. i see the socket connection closed in the ES logs.. so ES is
contacted ... the 9200 is correct ?

philippe
best regards

configuration.set("es.resource", "eshindex2/eshlog2/_search?q=*");
    configuration.set("es.host", "localhost");
    configuration.set("es.port", "9200");
    System.out.println("conf:"+ configuration);
    JobConf job = new JobConf(configuration, 

SimpleJobES2Hadoop.class);
...................

On Tuesday, February 4, 2014 11:07:37 AM UTC+1, Costin Leau wrote:

If you are using M1 you should use es.resource instead of es.query in your
latest example.
es.query is supported in the upcoming 1.3.0.M2 (not in m1).

On 04/02/2014 12:01 PM, Phil gib wrote:

hello my context:
ES 0.90.3, elasticsearch-hadoop-1.3.0.M1 , eclipse- java
i am experimenting ES->Hadoop and Hadoop->ES
no problem with Hadoop2ES with these 3 settings ( i see the index +
data through head plugin, perfect)
job.set("es.resource", "eshindex2/eshlog2");
job.set("es.host", "localhost");
job.set("es.port", "9200");
but experimenting the inverse , ie ES 2 Hadoop
....
configuration.set("es.query", "eshindex2/eshlog2/_search?q=*");
configuration.set("es.host", "localhost");
configuration.set("es.port", "9200");
System.out.println("conf:"+ configuration);
JobConf job = new JobConf(configuration,
SimpleJobES2Hadoop.class);
...................
i got the error
10:58:27 INFO mapred.JobClient: Cleaning up the staging area

file:/tmp/hadoop-xxx/mapred/staging/xxx-1302405475/.staging/job_local_0001

Exception in thread "main" java.lang.NullPointerException
at org.elasticsearch.hadoop.rest.dto.Node.(Node.java:30)
at
org.elasticsearch.hadoop.rest.RestClient.getNodes(RestClient.java:139)

Any idea?
bestsregards
philippe

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1353d6b0-830e-4881-b10d-533639227a1c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Costin Leau) #4

The port and host are 9200 and localhost by default. What's your ES version?
ES-Hadoop requires 0.90 or higher (preferably the latest 0.90).

On 04/02/2014 12:59 PM, Phil gib wrote:

Hello Costin,

changing to "es.resource" does not help... :frowning:
argh.. i see the socket connection closed in the ES logs.. so ES is contacted ... the 9200 is correct ?

philippe
best regards

 configuration.set("es.resource", "eshindex2/eshlog2/_search?q=*");
     configuration.set("es.host", "localhost");
     configuration.set("es.port", "9200");
     System.out.println("conf:"+ configuration);
     JobConf job = new JobConf(configuration, SimpleJobES2Hadoop.class);

...................

On Tuesday, February 4, 2014 11:07:37 AM UTC+1, Costin Leau wrote:

If you are using M1 you should use es.resource instead of es.query in your latest example.
es.query is supported in the upcoming 1.3.0.M2 (not in m1).

On 04/02/2014 12:01 PM, Phil gib wrote:
> hello my context:
>   ES 0.90.3,   elasticsearch-hadoop-1.3.0.M1 , eclipse- java
> i am experimenting ES->Hadoop  and Hadoop->ES
> no problem with Hadoop2ES  with these 3 settings  ( i see the index + data  through head plugin, perfect)
> job.set("es.resource", "eshindex2/eshlog2");
> job.set("es.host", "localhost");
> job.set("es.port", "9200");
> but experimenting the inverse , ie ES 2 Hadoop
>         ....
>          configuration.set("es.query", "eshindex2/eshlog2/_search?q=*");
>          configuration.set("es.host", "localhost");
>          configuration.set("es.port", "9200");
>          System.out.println("conf:"+ configuration);
>          JobConf job = new JobConf(configuration, SimpleJobES2Hadoop.class);
> ...................
> i got the error
> 10:58:27 INFO mapred.JobClient: Cleaning up the staging area
> file:/tmp/hadoop-xxx/mapred/staging/xxx-1302405475/.staging/job_local_0001
> Exception in thread "main" java.lang.NullPointerException
>      at org.elasticsearch.hadoop.rest.dto.Node.<init>(Node.java:30)
>      at org.elasticsearch.hadoop.rest.RestClient.getNodes(RestClient.java:139)
>
>
> Any idea?
> bestsregards
> philippe
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
>elasticsearc...@googlegroups.com <javascript:>.
> To view this discussion on the web visit
>https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com>.
> For more options, visithttps://groups.google.com/groups/opt_out <https://groups.google.com/groups/opt_out>.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1353d6b0-830e-4881-b10d-533639227a1c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/52F0CD8E.30100%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(philg) #5

Thnaks for your reply,
my SW context is:
ES 0.90.3, elasticsearch-hadoop-1.3.0.M1 , eclipse- java, windows7

(perharps some configuration problem in Eclipse , i will investigate )
best regards
philippe

On Tuesday, February 4, 2014 12:22:54 PM UTC+1, Costin Leau wrote:

The port and host are 9200 and localhost by default. What's your ES
version?
ES-Hadoop requires 0.90 or higher (preferably the latest 0.90).

On 04/02/2014 12:59 PM, Phil gib wrote:

Hello Costin, 

changing to "es.resource" does not help... :frowning:
argh.. i see the socket connection closed in the ES logs.. so ES is
contacted ... the 9200 is correct ?

philippe
best regards

 configuration.set("es.resource", "eshindex2/eshlog2/_search?q=*"); 
     configuration.set("es.host", "localhost"); 
     configuration.set("es.port", "9200"); 
     System.out.println("conf:"+ configuration); 
     JobConf job = new JobConf(configuration, 

SimpleJobES2Hadoop.class);

...................

On Tuesday, February 4, 2014 11:07:37 AM UTC+1, Costin Leau wrote:

If you are using M1 you should use es.resource instead of es.query 

in your latest example.

es.query is supported in the upcoming 1.3.0.M2 (not in m1). 

On 04/02/2014 12:01 PM, Phil gib wrote: 
> hello my context: 
>   ES 0.90.3,   elasticsearch-hadoop-1.3.0.M1 , eclipse- java 
> i am experimenting ES->Hadoop  and Hadoop->ES 
> no problem with Hadoop2ES  with these 3 settings  ( i see the 

index + data through head plugin, perfect)

> job.set("es.resource", "eshindex2/eshlog2"); 
> job.set("es.host", "localhost"); 
> job.set("es.port", "9200"); 
> but experimenting the inverse , ie ES 2 Hadoop 
>         .... 
>          configuration.set("es.query", 

"eshindex2/eshlog2/_search?q=*");

>          configuration.set("es.host", "localhost"); 
>          configuration.set("es.port", "9200"); 
>          System.out.println("conf:"+ configuration); 
>          JobConf job = new JobConf(configuration, 

SimpleJobES2Hadoop.class);

> ................... 
> i got the error 
> 10:58:27 INFO mapred.JobClient: Cleaning up the staging area 
> 

file:/tmp/hadoop-xxx/mapred/staging/xxx-1302405475/.staging/job_local_0001

> Exception in thread "main" java.lang.NullPointerException 
>      at 

org.elasticsearch.hadoop.rest.dto.Node.(Node.java:30)

>      at 

org.elasticsearch.hadoop.rest.RestClient.getNodes(RestClient.java:139)

> 
> 
> Any idea? 
> bestsregards 
> philippe 
> 
> -- 
> You received this message because you are subscribed to the Google 

Groups "elasticsearch" group.

> To unsubscribe from this group and stop receiving emails from it, 

send an email to

>elasticsearc...@googlegroups.com <javascript:>. 
> To view this discussion on the web visit 
>

https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/2a52b4e7-3ecd-491b-8c6e-1d8d521260ef%40googlegroups.com>.

> For more options, visithttps://groups.google.com/groups/opt_out <

https://groups.google.com/groups/opt_out>.

-- 
Costin 

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/1353d6b0-830e-4881-b10d-533639227a1c%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8f8fbb72-097d-47e8-8c2c-d2f74a1f4b00%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6