How can I use elasticsearch-hadoop plugin?

Hi all,
This question may be too easy, but I am a new in elasticsearch. so.....
I want to use elasticsearch-hadoop plugin, but it is not working. Here
is my code.

public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
String query = "twitter/tweet/_search?routing=kimchy";

    configuration.set("es.resource", query);
    
    Job job = new Job(configuration);
    Path out = new Path(args[0]);

    job.setInputFormatClass(ESInputFormat.class);
    job.setOutputFormatClass(ESOutputFormat.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(MapWritable.class);

    job.waitForCompletion(true);

}

I just want to know how to use this plugin.............

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hello Hoony,

I noticed you posted a few times without getting any responses. Not many
people, including me, use the Hadoop plugin, so getting help might be
difficult.

The one thing that sticks out from your example is that your search
explicitly specifies that routing values (routing=kimchy). Can you retry
your example without it?

--
Ivan

On Wed, May 22, 2013 at 1:12 AM, Hoony monkeyboywon@gmail.com wrote:

Hi all,
This question may be too easy, but I am a new in elasticsearch. so.....
I want to use elasticsearch-hadoop plugin, but it is not working. Here
is my code.

public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
String query = "twitter/tweet/_search?routing=kimchy";

    configuration.set("es.resource", query);

    Job job = new Job(configuration);
    Path out = new Path(args[0]);

    job.setInputFormatClass(ESInputFormat.class);
    job.setOutputFormatClass(ESOutputFormat.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(MapWritable.class);

    job.waitForCompletion(true);

}

I just want to know how to use this plugin.............

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Thank you for your response :slight_smile:

Do you mean run my example like here?

public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
String query = "twitter/tweet/_search?"; <======== String query
= "twitter/tweet/_search?"

    configuration.set("es.resource", query);
    
    Job job = new Job(configuration);
    Path out = new Path(args[0]);

    job.setInputFormatClass(ESInputFormat.class);
    job.setOutputFormatClass(ESOutputFormat.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(MapWritable.class);

    job.waitForCompletion(true);

}

It appear the same result. The 500 status was returned. And here is a more
detail description about my issue.
https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/RfnT5kst8Gc

在 2013年5月24日星期五UTC+9上午3时56分02秒,Ivan Brusic写道:

Hello Hoony,

I noticed you posted a few times without getting any responses. Not many
people, including me, use the Hadoop plugin, so getting help might be
difficult.

The one thing that sticks out from your example is that your search
explicitly specifies that routing values (routing=kimchy). Can you retry
your example without it?

--
Ivan

On Wed, May 22, 2013 at 1:12 AM, Hoony <monkey...@gmail.com <javascript:>>wrote:

Hi all,
This question may be too easy, but I am a new in elasticsearch. so.....
I want to use elasticsearch-hadoop plugin, but it is not working. Here
is my code.

public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
String query = "twitter/tweet/_search?routing=kimchy";

    configuration.set("es.resource", query);
    
    Job job = new Job(configuration);
    Path out = new Path(args[0]);

    job.setInputFormatClass(ESInputFormat.class);
    job.setOutputFormatClass(ESOutputFormat.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(MapWritable.class);

    job.waitForCompletion(true);

}

I just want to know how to use this plugin.............

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,

Sorry I didn't pick on this thread earlier - it must have slipped under my radar.
Make sure you're using Elasticsearch 0.90 (see the readme) or higher - the integration uses specific features added in
0.90 and doesn't work on lower versions.

Let me know if you have any issues.

P.S. To simplify the initial deployment use a locally configured Hadoop so everything runs within the same JVM and you
don't have classpath issues. If you want to use a remote VM/cluster then make sure you add elasticsearch-hadoop jar to
you job classpath.

Cheers,

On 28/05/2013 4:49 AM, Hoony wrote:

Thank you for your response :slight_smile:

Do you mean run my example like here?

public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
String query = "twitter/tweet/_search?"; <======== String query = "twitter/tweet/_search?"

     configuration.set("es.resource", query);

     Job job = new Job(configuration);
     Path out = new Path(args[0]);

     job.setInputFormatClass(ESInputFormat.class);
     job.setOutputFormatClass(ESOutputFormat.class);
     job.setOutputKeyClass(Text.class);
     job.setOutputValueClass(MapWritable.class);

     job.waitForCompletion(true);

}

It appear the same result. The 500 status was returned. And here is a more detail description about my issue.
https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/RfnT5kst8Gc

在 2013年5月24日星期五UTC+9上午3时56分02秒,Ivan Brusic写道:

Hello Hoony,

I noticed you posted a few times without getting any responses. Not many people, including me, use the Hadoop
plugin, so getting help might be difficult.

The one thing that sticks out from your example is that your search explicitly specifies that routing values
(routing=kimchy). Can you retry your example without it?

--
Ivan


On Wed, May 22, 2013 at 1:12 AM, Hoony <monkey...@gmail.com <javascript:>> wrote:

    Hi all,
        This question may be too easy, but I am a new in elasticsearch. so.....
        I want to use elasticsearch-hadoop plugin, but it is not working. Here is my code.



    public static void main(String[] args) throws Exception {
             Configuration configuration = new Configuration();
             String query = "twitter/tweet/_search?routing=kimchy";

             configuration.set("es.resource", query);

             Job job = new Job(configuration);
             Path out = new Path(args[0]);

             job.setInputFormatClass(ESInputFormat.class);
             job.setOutputFormatClass(ESOutputFormat.class);
             job.setOutputKeyClass(Text.class);
             job.setOutputValueClass(MapWritable.class);

             job.waitForCompletion(true);
       }


    I just want to know how to use this plugin.............

    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearc...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/groups/opt_out <https://groups.google.com/groups/opt_out>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Thank you for your reply :slight_smile:

I checked elasticsearch version which I used, and it is version 9.0.
All test environment is in localhost. Would you mind look at the web page
which I describe more detail? ----
https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/how$20to$20use$20elasticsearch-hadoop/elasticsearch/RfnT5kst8Gc/wkV7iF_Kve8J
https://lh6.googleusercontent.com/-hPlX2xSV9Jg/UaQ9gQSoYnI/AAAAAAAAADw/HYnhLmQtwUg/s1600/list.gif

在 2013年5月28日星期二UTC+9下午12时48分06秒,Costin Leau写道:

Hi,

Sorry I didn't pick on this thread earlier - it must have slipped under my
radar.
Make sure you're using Elasticsearch 0.90 (see the readme) or higher - the
integration uses specific features added in
0.90 and doesn't work on lower versions.

Let me know if you have any issues.

P.S. To simplify the initial deployment use a locally configured Hadoop so
everything runs within the same JVM and you
don't have classpath issues. If you want to use a remote VM/cluster then
make sure you add elasticsearch-hadoop jar to
you job classpath.

Cheers,

On 28/05/2013 4:49 AM, Hoony wrote:

Thank you for your response :slight_smile:

Do you mean run my example like here?

public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
String query = "twitter/tweet/_search?"; <======== String query =
"twitter/tweet/_search?"

     configuration.set("es.resource", query); 

     Job job = new Job(configuration); 
     Path out = new Path(args[0]); 

     job.setInputFormatClass(ESInputFormat.class); 
     job.setOutputFormatClass(ESOutputFormat.class); 
     job.setOutputKeyClass(Text.class); 
     job.setOutputValueClass(MapWritable.class); 

     job.waitForCompletion(true); 

}

It appear the same result. The 500 status was returned. And here is a
more detail description about my issue.

https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/RfnT5kst8Gc

在 2013年5月24日星期五UTC+9上午3时56分02秒,Ivan Brusic写道:

Hello Hoony, 

I noticed you posted a few times without getting any responses. Not 

many people, including me, use the Hadoop

plugin, so getting help might be difficult. 

The one thing that sticks out from your example is that your search 

explicitly specifies that routing values

(routing=kimchy). Can you retry your example without it? 

-- 
Ivan 


On Wed, May 22, 2013 at 1:12 AM, Hoony <monkey...@gmail.com<javascript:>> wrote: 

    Hi all, 
        This question may be too easy, but I am a new in 

elasticsearch. so.....

        I want to use elasticsearch-hadoop plugin, but it is not 

working. Here is my code.

    public static void main(String[] args) throws Exception { 
             Configuration configuration = new Configuration(); 
             String query = "twitter/tweet/_search?routing=kimchy"; 

             configuration.set("es.resource", query); 

             Job job = new Job(configuration); 
             Path out = new Path(args[0]); 

             job.setInputFormatClass(ESInputFormat.class); 
             job.setOutputFormatClass(ESOutputFormat.class); 
             job.setOutputKeyClass(Text.class); 
             job.setOutputValueClass(MapWritable.class); 

             job.waitForCompletion(true); 
       } 


    I just want to know how to use this plugin............. 

    -- 
    You received this message because you are subscribed to the 

Google Groups "elasticsearch" group.

    To unsubscribe from this group and stop receiving emails from 

it, send an email to

    elasticsearc...@googlegroups.com <javascript:>. 
    For more options, visit https://groups.google.com/groups/opt_out<

https://groups.google.com/groups/opt_out>.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Thank you for your reply :slight_smile:

I checked elasticsearch version which I used, and it is version 9.0.
All test environment is in localhost. Would you mind look at the web page
which I describe more detail? ----
https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/how$20to$20use$20elasticsearch-hadoop/elasticsearch/RfnT5kst8Gc/wkV7iF_Kve8J
https://lh6.googleusercontent.com/-hPlX2xSV9Jg/UaQ9gQSoYnI/AAAAAAAAADw/HYnhLmQtwUg/s1600/list.gif

在 2013年5月28日星期二UTC+9下午12时48分06秒,Costin Leau写道:

Hi,

Sorry I didn't pick on this thread earlier - it must have slipped under my
radar.
Make sure you're using Elasticsearch 0.90 (see the readme) or higher - the
integration uses specific features added in
0.90 and doesn't work on lower versions.

Let me know if you have any issues.

P.S. To simplify the initial deployment use a locally configured Hadoop so
everything runs within the same JVM and you
don't have classpath issues. If you want to use a remote VM/cluster then
make sure you add elasticsearch-hadoop jar to
you job classpath.

Cheers,

On 28/05/2013 4:49 AM, Hoony wrote:

Thank you for your response :slight_smile:

Do you mean run my example like here?

public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
String query = "twitter/tweet/_search?"; <======== String query =
"twitter/tweet/_search?"

     configuration.set("es.resource", query); 

     Job job = new Job(configuration); 
     Path out = new Path(args[0]); 

     job.setInputFormatClass(ESInputFormat.class); 
     job.setOutputFormatClass(ESOutputFormat.class); 
     job.setOutputKeyClass(Text.class); 
     job.setOutputValueClass(MapWritable.class); 

     job.waitForCompletion(true); 

}

It appear the same result. The 500 status was returned. And here is a
more detail description about my issue.

https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/RfnT5kst8Gc

在 2013年5月24日星期五UTC+9上午3时56分02秒,Ivan Brusic写道:

Hello Hoony, 

I noticed you posted a few times without getting any responses. Not 

many people, including me, use the Hadoop

plugin, so getting help might be difficult. 

The one thing that sticks out from your example is that your search 

explicitly specifies that routing values

(routing=kimchy). Can you retry your example without it? 

-- 
Ivan 


On Wed, May 22, 2013 at 1:12 AM, Hoony <monkey...@gmail.com<javascript:>> wrote: 

    Hi all, 
        This question may be too easy, but I am a new in 

elasticsearch. so.....

        I want to use elasticsearch-hadoop plugin, but it is not 

working. Here is my code.

    public static void main(String[] args) throws Exception { 
             Configuration configuration = new Configuration(); 
             String query = "twitter/tweet/_search?routing=kimchy"; 

             configuration.set("es.resource", query); 

             Job job = new Job(configuration); 
             Path out = new Path(args[0]); 

             job.setInputFormatClass(ESInputFormat.class); 
             job.setOutputFormatClass(ESOutputFormat.class); 
             job.setOutputKeyClass(Text.class); 
             job.setOutputValueClass(MapWritable.class); 

             job.waitForCompletion(true); 
       } 


    I just want to know how to use this plugin............. 

    -- 
    You received this message because you are subscribed to the 

Google Groups "elasticsearch" group.

    To unsubscribe from this group and stop receiving emails from 

it, send an email to

    elasticsearc...@googlegroups.com <javascript:>. 
    For more options, visit https://groups.google.com/groups/opt_out<

https://groups.google.com/groups/opt_out>.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Your query is incorrect - don't use routing (we don't support it yet and we
might even forbid it since the runtime already does 'routing' creating
multiple Hadoop splits per available shards).
Start with a basic query such as : "twitter/tweet/search?q=*" and take it
from there.

Currently the parsing is of the query string does not cover all the cases
but will be addressed in the near future.

Hth,

On Wed, May 29, 2013 at 10:46 AM, Hoony monkeyboywon@gmail.com wrote:

Thank you for your reply :slight_smile:

I checked elasticsearch version which I used, and it is version 9.0.
All test environment is in localhost. Would you mind look at the web page
which I describe more detail? ---- https://groups.google.com/**
forum/?fromgroups#!searchin/elasticsearch/how$20to$20use$
20elasticsearch-hadoop/**elasticsearch/RfnT5kst8Gc/**wkV7iF_Kve8Jhttps://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/how$20to$20use$20elasticsearch-hadoop/elasticsearch/RfnT5kst8Gc/wkV7iF_Kve8J

https://lh6.googleusercontent.com/-hPlX2xSV9Jg/UaQ9gQSoYnI/AAAAAAAAADw/HYnhLmQtwUg/s1600/list.gif

在 2013年5月28日星期二UTC+9下午12时48分06秒,Costin Leau写道:

Hi,

Sorry I didn't pick on this thread earlier - it must have slipped under
my radar.
Make sure you're using Elasticsearch 0.90 (see the readme) or higher -
the integration uses specific features added in
0.90 and doesn't work on lower versions.

Let me know if you have any issues.

P.S. To simplify the initial deployment use a locally configured Hadoop
so everything runs within the same JVM and you
don't have classpath issues. If you want to use a remote VM/cluster then
make sure you add elasticsearch-hadoop jar to
you job classpath.

Cheers,

On 28/05/2013 4:49 AM, Hoony wrote:

Thank you for your response :slight_smile:

Do you mean run my example like here?

public static void main(String[] args) throws Exception {
Configuration configuration = new Configuration();
String query = "twitter/tweet/_search?"; <======== String query =
"twitter/tweet/_search?"

     configuration.set("es.**resource", query);

     Job job = new Job(configuration);
     Path out = new Path(args[0]);

     job.setInputFormatClass(**ESInputFormat.class);
     job.setOutputFormatClass(**ESOutputFormat.class);
     job.setOutputKeyClass(Text.**class);
     job.setOutputValueClass(**MapWritable.class);

     job.waitForCompletion(true);

}

It appear the same result. The 500 status was returned. And here is a
more detail description about my issue.
https://groups.google.com/forum/?fromgroups#!topic/
elasticsearch/RfnT5kst8Gchttps://groups.google.com/forum/?fromgroups#!topic/elasticsearch/RfnT5kst8Gc

在 2013年5月24日星期五UTC+9上午3时56分02秒,**Ivan Brusic写道:

Hello Hoony,

I noticed you posted a few times without getting any responses. Not

many people, including me, use the Hadoop

plugin, so getting help might be difficult.

The one thing that sticks out from your example is that your search

explicitly specifies that routing values

(routing=kimchy). Can you retry your example without it?

--
Ivan


On Wed, May 22, 2013 at 1:12 AM, Hoony <monkey...@gmail.com<javascript:>> wrote:

    Hi all,
        This question may be too easy, but I am a new in

elasticsearch. so.....

        I want to use elasticsearch-hadoop plugin, but it is not

working. Here is my code.

    public static void main(String[] args) throws Exception {
             Configuration configuration = new Configuration();
             String query = "twitter/tweet/_search?**routing=kimchy";
             configuration.set("es.**resource", query);

             Job job = new Job(configuration);
             Path out = new Path(args[0]);

             job.setInputFormatClass(**ESInputFormat.class);
             job.setOutputFormatClass(**ESOutputFormat.class);
             job.setOutputKeyClass(Text.**class);
             job.setOutputValueClass(**MapWritable.class);

             job.waitForCompletion(true);
       }


    I just want to know how to use this plugin.............

    --
    You received this message because you are subscribed to the

Google Groups "elasticsearch" group.

    To unsubscribe from this group and stop receiving emails from

it, send an email to

    elasticsearc...@googlegroups.**com <javascript:>.
    For more options, visit https://groups.google.com/**

groups/opt_out https://groups.google.com/groups/opt_out <
https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out>.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.