Error while reading elasticsearch data in hadoop program

Hi, I was following the example given on official elasticsearch
documentation to read data from elasticsearch using hadoop but i am getting
the following error.

java.lang.Exception: java.io.IOException: Type mismatch in value from map:
expected org.apache.hadoop.io.MapWritable, received
org.elasticsearch.hadoop.mr.LinkedMapWritable

at
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)

Caused by: java.io.IOException: Type mismatch in value from map: expected
org.apache.hadoop.io.MapWritable, received
org.elasticsearch.hadoop.mr.LinkedMapWritable

at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1054)

at
org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:587)

at es2.mapper1.map(mapper1.java:29)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)

at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

at java.util.concurrent.FutureTask.run(FutureTask.java:166)

at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:724)

14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed
with state FAILED due to: NA

14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0

Exception in thread "main" java.io.IOException: Job failed!

at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)

at es2.Es2.main(Es2.java:48)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
[node=[QMYbfSRMTsyXVVgfFweMkA/Conceptnet|192.168.1.202:9200],shard=0] > map

The code for my main runner class is as follows:-

public class Es2 {

    static private final Path TMP_DIR = new 

Path(Es2.class.getSimpleName()

        + "_TMP_1");


/**

 * @param args the command line arguments

 */

public static void main(String[] args) throws IOException{

//#String node="192.168.1.202";

JobConf conf = new JobConf();

conf.set("es.resource", "data/content");   

conf.set("es.nodes", "192.168.1.202:9200"); 

conf.set("es.query", "?q=s_start:java");

conf.setInputFormat(EsInputFormat.class);       

conf.setMapOutputKeyClass(Text.class);          

conf.setMapOutputValueClass(MapWritable.class);

conf.setOutputKeyClass(Text.class);

conf.setOutputValueClass(MapWritable.class);

conf.setOutputFormat(TextOutputFormat.class);

conf.setMapperClass(mapper1.class);

final Path outDir = new Path(TMP_DIR, "out");

FileOutputFormat.setOutputPath(conf, outDir);

JobClient.runJob(conf);

}

}

The code for my mapper class is as follows:-

public class mapper1 extends MapReduceBase implements

    Mapper{

@Override

public void map(Object key, Object value, OutputCollector output,

                Reporter reporter) throws IOException {

Text docId = (Text) key;

MapWritable doc = (MapWritable) value;

output.collect(docId,doc);

}

}

Kindly guide me on this issue.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi,

What version of es-hadoop are you using? The problem stems from the difference in the types mentioned on your Mapper,
namely the output value class:

 conf.setMapOutputValueClass(MapWritable.class);

to MapWritable while LinkedMapWritable is returned. The latest versions automatically detect this and use the proper
type so I recommend upgrading.
If that's not an option, use LinkedMapWritable.

Cheers,

P.S. Please don't post code and stracktraces on the mailing list since it highly reduces the readability of your email.
Instead use gist or any other service
to post the code as indicated in the docs [1]. Thanks

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/troubleshooting.html#_where_do_i_post_my_information

On 9/9/14 11:59 AM, gaurav redkar wrote:

Hi, I was following the example given on official elasticsearch documentation to read data from elasticsearch using
hadoop but i am getting the following error.

java.lang.Exception: java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.MapWritable,
received org.elasticsearch.hadoop.mr.LinkedMapWritable

at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)

Caused by: java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.MapWritable, received
org.elasticsearch.hadoop.mr.LinkedMapWritable

at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1054)

at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:587)

at es2.mapper1.map(mapper1.java:29)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)

at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

at java.util.concurrent.FutureTask.run(FutureTask.java:166)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:724)

14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed with state FAILED due to: NA

14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0

Exception in thread "main" java.io.IOException: Job failed!

at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)

at es2.Es2.main(Es2.java:48)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
[node=[QMYbfSRMTsyXVVgfFweMkA/Conceptnet|192.168.1.202:9200],shard=0] > map

The code for my main runner class is as follows:-

public class Es2 {

     static private final Path TMP_DIR = new Path(Es2.class.getSimpleName()

         + "_TMP_1");


 /**

  * @param args the command line arguments

  */

 public static void main(String[] args) throws IOException{

//#String node="192.168.1.202";

 JobConf conf = new JobConf();

 conf.set("es.resource", "data/content");

 conf.set("es.nodes", "192.168.1.202:9200");

 conf.set("es.query", "?q=s_start:java");

 conf.setInputFormat(EsInputFormat.class);

 conf.setMapOutputKeyClass(Text.class);

 conf.setMapOutputValueClass(MapWritable.class);

 conf.setOutputKeyClass(Text.class);

 conf.setOutputValueClass(MapWritable.class);

 conf.setOutputFormat(TextOutputFormat.class);

 conf.setMapperClass(mapper1.class);

 final Path outDir = new Path(TMP_DIR, "out");

 FileOutputFormat.setOutputPath(conf, outDir);

 JobClient.runJob(conf);

 }

}

The code for my mapper class is as follows:-

public class mapper1 extends MapReduceBase implements

     Mapper{

@Override

public void map(Object key, Object value, OutputCollector output,

                 Reporter reporter) throws IOException {

Text docId = (Text) key;

MapWritable doc = (MapWritable) value;

output.collect(docId,doc);

}

}

Kindly guide me on this issue.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/540ED720.5040501%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Hi Costin,

Thanks for the heads up regarding gist. I will try to follow the guidelines
in the future. As for my program, I am using Elasticsearch Hadoop
v2.1.0.Beta1 . I tried your suggestion and changed the output value class
to LinkedMapWritable. but now I am getting the following error.

Any idea why is this happening ? I even tried using the v2.0.0 of es-hadoop
but am still getting the same error.

On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau costin.leau@gmail.com wrote:

Hi,

What version of es-hadoop are you using? The problem stems from the
difference in the types mentioned on your Mapper, namely the output value
class:

  conf.setMapOutputValueClass(MapWritable.class);

to MapWritable while LinkedMapWritable is returned. The latest versions
automatically detect this and use the proper type so I recommend upgrading.
If that's not an option, use LinkedMapWritable.

Cheers,

P.S. Please don't post code and stracktraces on the mailing list since it
highly reduces the readability of your email. Instead use gist or any other
service
to post the code as indicated in the docs [1]. Thanks

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/
2.1.Beta/troubleshooting.html#_where_do_i_post_my_information

On 9/9/14 11:59 AM, gaurav redkar wrote:

Hi, I was following the example given on official elasticsearch
documentation to read data from elasticsearch using
hadoop but i am getting the following error.

java.lang.Exception: java.io.IOException: Type mismatch in value from
map: expected org.apache.hadoop.io.MapWritable,
received org.elasticsearch.hadoop.mr.LinkedMapWritable

at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(
LocalJobRunner.java:462)

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(
LocalJobRunner.java:522)

Caused by: java.io.IOException: Type mismatch in value from map: expected
org.apache.hadoop.io.MapWritable, received
org.elasticsearch.hadoop.mr.LinkedMapWritable

at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.
collect(MapTask.java:1054)

at org.apache.hadoop.mapred.MapTask$OldOutputCollector.
collect(MapTask.java:587)

at es2.mapper1.map(mapper1.java:29)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)

at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(
LocalJobRunner.java:243)

at java.util.concurrent.Executors$RunnableAdapter.
call(Executors.java:471)

at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

at java.util.concurrent.FutureTask.run(FutureTask.java:166)

at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:724)

14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed
with state FAILED due to: NA

14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0

Exception in thread "main" java.io.IOException: Job failed!

at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)

at es2.Es2.main(Es2.java:48)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
[node=[QMYbfSRMTsyXVVgfFweMkA/Conceptnet|192.168.1.202:9200],shard=0] >
map

The code for my main runner class is as follows:-

public class Es2 {

     static private final Path TMP_DIR = new

Path(Es2.class.getSimpleName()

         + "_TMP_1");


 /**

  * @param args the command line arguments

  */

 public static void main(String[] args) throws IOException{

//#String node="192.168.1.202";

 JobConf conf = new JobConf();

 conf.set("es.resource", "data/content");

 conf.set("es.nodes", "192.168.1.202:9200");

 conf.set("es.query", "?q=s_start:java");

 conf.setInputFormat(EsInputFormat.class);

 conf.setMapOutputKeyClass(Text.class);

 conf.setMapOutputValueClass(MapWritable.class);

 conf.setOutputKeyClass(Text.class);

 conf.setOutputValueClass(MapWritable.class);

 conf.setOutputFormat(TextOutputFormat.class);

 conf.setMapperClass(mapper1.class);

 final Path outDir = new Path(TMP_DIR, "out");

 FileOutputFormat.setOutputPath(conf, outDir);

 JobClient.runJob(conf);

 }

}

The code for my mapper class is as follows:-

public class mapper1 extends MapReduceBase implements

     Mapper{

@Override

public void map(Object key, Object value, OutputCollector output,

                 Reporter reporter) throws IOException {

Text docId = (Text) key;

MapWritable doc = (MapWritable) value;

output.collect(docId,doc);

}

}

Kindly guide me on this issue.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3b110c43-
faa8-4a08-acfa-0fac5fd89585%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-
faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=
email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/HS1A_psM7fw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/540ED720.5040501%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Most likely you have a classpath conflict caused by multiple versions of es-hadoop. Can you double check you only have
one version (2.1.0.Beta1) available?
Based on the error, I'm guessing you have some 1.3 Mx or the RC somewhere in there...

On 9/9/14 4:06 PM, gaurav redkar wrote:

Hi Costin,

Thanks for the heads up regarding gist. I will try to follow the guidelines in the future. As for my program, I am using
Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and changed the output value class to LinkedMapWritable. but
now I am getting the following error.

https://gist.github.com/gauravub/7d55bc6b10cb63935eb8

Any idea why is this happening ? I even tried using the v2.0.0 of es-hadoop but am still getting the same error.

On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com> wrote:

Hi,

What version of es-hadoop are you using? The problem stems from the difference in the types mentioned on your
Mapper, namely the output value class:

          conf.setMapOutputValueClass(__MapWritable.class);


to MapWritable while LinkedMapWritable is returned. The latest versions automatically detect this and use the proper
type so I recommend upgrading.
If that's not an option, use LinkedMapWritable.

Cheers,

P.S. Please don't post code and stracktraces on the mailing list since it highly reduces the readability of your
email. Instead use gist or any other service
to post the code as indicated in the docs [1]. Thanks

[1]
http://www.elasticsearch.org/__guide/en/elasticsearch/hadoop/__2.1.Beta/troubleshooting.html#___where_do_i_post_my___information
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/troubleshooting.html#_where_do_i_post_my_information>




On 9/9/14 11:59 AM, gaurav redkar wrote:

    Hi, I was following the example given on official elasticsearch documentation to read data from elasticsearch using
    hadoop but i am getting the following error.

    java.lang.Exception: java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io
    <http://org.apache.hadoop.io>.__MapWritable,
    received org.elasticsearch.hadoop.mr <http://org.elasticsearch.hadoop.mr>.__LinkedMapWritable

    at org.apache.hadoop.mapred.__LocalJobRunner$Job.runTasks(__LocalJobRunner.java:462)

    at org.apache.hadoop.mapred.__LocalJobRunner$Job.run(__LocalJobRunner.java:522)

    Caused by: java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io
    <http://org.apache.hadoop.io>.__MapWritable, received
    org.elasticsearch.hadoop.mr <http://org.elasticsearch.hadoop.mr>.__LinkedMapWritable

    at org.apache.hadoop.mapred.__MapTask$MapOutputBuffer.__collect(MapTask.java:1054)

    at org.apache.hadoop.mapred.__MapTask$OldOutputCollector.__collect(MapTask.java:587)

    at es2.mapper1.map(mapper1.java:__29)

    at org.apache.hadoop.mapred.__MapRunner.run(MapRunner.java:__54)

    at org.apache.hadoop.mapred.__MapTask.runOldMapper(MapTask.__java:430)

    at org.apache.hadoop.mapred.__MapTask.run(MapTask.java:342)

    at org.apache.hadoop.mapred.__LocalJobRunner$Job$__MapTaskRunnable.run(__LocalJobRunner.java:243)

    at java.util.concurrent.__Executors$RunnableAdapter.__call(Executors.java:471)

    at java.util.concurrent.__FutureTask$Sync.innerRun(__FutureTask.java:334)

    at java.util.concurrent.__FutureTask.run(FutureTask.__java:166)

    at java.util.concurrent.__ThreadPoolExecutor.runWorker(__ThreadPoolExecutor.java:1145)

    at java.util.concurrent.__ThreadPoolExecutor$Worker.run(__ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.__java:724)

    14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed with state FAILED due to: NA

    14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0

    Exception in thread "main" java.io.IOException: Job failed!

    at org.apache.hadoop.mapred.__JobClient.runJob(JobClient.__java:836)

    at es2.Es2.main(Es2.java:48)

    at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native Method)

    at sun.reflect.__NativeMethodAccessorImpl.__invoke(__NativeMethodAccessorImpl.java:__57)

    at sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)

    at java.lang.reflect.Method.__invoke(Method.java:606)

    at org.apache.hadoop.util.RunJar.__main(RunJar.java:212)

    14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
    [node=[QMYbfSRMTsyXVVgfFweMkA/__Conceptnet|192.168.1.202:9200 <http://192.168.1.202:9200>]__,shard=0] > map


    The code for my main runner class is as follows:-


    public class Es2 {

              static private final Path TMP_DIR = new Path(Es2.class.getSimpleName()

                  + "_TMP_1");


          /**

           * @param args the command line arguments

           */

          public static void main(String[] args) throws IOException{

         //#String node="192.168.1.202";

          JobConf conf = new JobConf();

          conf.set("es.resource", "data/content");

          conf.set("es.nodes", "192.168.1.202:9200 <http://192.168.1.202:9200>");

          conf.set("es.query", "?q=s_start:java");

          conf.setInputFormat(__EsInputFormat.class);

          conf.setMapOutputKeyClass(__Text.class);

          conf.setMapOutputValueClass(__MapWritable.class);

          conf.setOutputKeyClass(Text.__class);

          conf.setOutputValueClass(__MapWritable.class);

          conf.setOutputFormat(__TextOutputFormat.class);

          conf.setMapperClass(mapper1.__class);

          final Path outDir = new Path(TMP_DIR, "out");

          FileOutputFormat.__setOutputPath(conf, outDir);

          JobClient.runJob(conf);

          }

    }

    The code for my mapper class is as follows:-

    public class mapper1 extends MapReduceBase implements

              Mapper{

       @Override

       public void map(Object key, Object value, OutputCollector output,

                          Reporter reporter) throws IOException {

         Text docId = (Text) key;

         MapWritable doc = (MapWritable) value;

         output.collect(docId,doc);

       }

    }

    Kindly guide me on this issue.

    Thanks

    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com>
    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=email&utm_source=footer>>.
    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_psM7fw/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/540ED720.__5040501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540ED720.5040501%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.
--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/540EFF05.6040508%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Hi Costin,

I had downloaded the elasticsearch-hadoop-2.1.0.Beta1.zip file and used all
the jars from that for the program. Later I even tried replacing all the
jars in my program with jars from with elasticsearch-hadoop-2.0.0.zip file,
but still facing the same error.

On Tue, Sep 9, 2014 at 6:52 PM, Costin Leau costin.leau@gmail.com wrote:

Most likely you have a classpath conflict caused by multiple versions of
es-hadoop. Can you double check you only have one version (2.1.0.Beta1)
available?
Based on the error, I'm guessing you have some 1.3 Mx or the RC somewhere
in there...

On 9/9/14 4:06 PM, gaurav redkar wrote:

Hi Costin,

Thanks for the heads up regarding gist. I will try to follow the
guidelines in the future. As for my program, I am using
Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and changed
the output value class to LinkedMapWritable. but
now I am getting the following error.

https://gist.github.com/gauravub/7d55bc6b10cb63935eb8

Any idea why is this happening ? I even tried using the v2.0.0 of
es-hadoop but am still getting the same error.

On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <costin.leau@gmail.com
mailto:costin.leau@gmail.com> wrote:

Hi,

What version of es-hadoop are you using? The problem stems from the

difference in the types mentioned on your
Mapper, namely the output value class:

          conf.setMapOutputValueClass(__MapWritable.class);


to MapWritable while LinkedMapWritable is returned. The latest

versions automatically detect this and use the proper
type so I recommend upgrading.
If that's not an option, use LinkedMapWritable.

Cheers,

P.S. Please don't post code and stracktraces on the mailing list

since it highly reduces the readability of your
email. Instead use gist or any other service
to post the code as indicated in the docs [1]. Thanks

[1]
http://www.elasticsearch.org/__guide/en/elasticsearch/

hadoop/__2.1.Beta/troubleshooting.html#__where
do_i_post_my___information
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/
2.1.Beta/troubleshooting.html#_where_do_i_post_my_information>

On 9/9/14 11:59 AM, gaurav redkar wrote:

    Hi, I was following the example given on official elasticsearch

documentation to read data from elasticsearch using
hadoop but i am getting the following error.

    java.lang.Exception: java.io.IOException: Type mismatch in value

from map: expected org.apache.hadoop.io
http://org.apache.hadoop.io.__MapWritable,
received org.elasticsearch.hadoop.mr <http://org.elasticsearch.
hadoop.mr>.__LinkedMapWritable

    at org.apache.hadoop.mapred.__LocalJobRunner$Job.runTasks(__

LocalJobRunner.java:462)

    at org.apache.hadoop.mapred.__LocalJobRunner$Job.run(__

LocalJobRunner.java:522)

    Caused by: java.io.IOException: Type mismatch in value from map:

expected org.apache.hadoop.io
http://org.apache.hadoop.io.__MapWritable, received
org.elasticsearch.hadoop.mr <http://org.elasticsearch.hadoop.mr

.__LinkedMapWritable

    at org.apache.hadoop.mapred.__MapTask$MapOutputBuffer.__

collect(MapTask.java:1054)

    at org.apache.hadoop.mapred.__MapTask$OldOutputCollector.__

collect(MapTask.java:587)

    at es2.mapper1.map(mapper1.java:__29)

    at org.apache.hadoop.mapred.__MapRunner.run(MapRunner.java:__54)

    at org.apache.hadoop.mapred.__MapTask.runOldMapper(MapTask._

_java:430)

    at org.apache.hadoop.mapred.__MapTask.run(MapTask.java:342)

    at org.apache.hadoop.mapred.__LocalJobRunner$Job$__

MapTaskRunnable.run(__LocalJobRunner.java:243)

    at java.util.concurrent.__Executors$RunnableAdapter.__

call(Executors.java:471)

    at java.util.concurrent.__FutureTask$Sync.innerRun(__

FutureTask.java:334)

    at java.util.concurrent.__FutureTask.run(FutureTask.__java:166)

    at java.util.concurrent.__ThreadPoolExecutor.runWorker(_

_ThreadPoolExecutor.java:1145)

    at java.util.concurrent.__ThreadPoolExecutor$Worker.run(

__ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.__java:724)

    14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001

failed with state FAILED due to: NA

    14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0

    Exception in thread "main" java.io.IOException: Job failed!

    at org.apache.hadoop.mapred.__JobClient.runJob(JobClient.__

java:836)

    at es2.Es2.main(Es2.java:48)

    at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native

Method)

    at sun.reflect.__NativeMethodAccessorImpl.__invoke(__

NativeMethodAccessorImpl.java:__57)

    at sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__

DelegatingMethodAccessorImpl.__java:43)

    at java.lang.reflect.Method.__invoke(Method.java:606)

    at org.apache.hadoop.util.RunJar.__main(RunJar.java:212)

    14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
    [node=[QMYbfSRMTsyXVVgfFweMkA/__Conceptnet|192.168.1.202:9200 <

http://192.168.1.202:9200>]__,shard=0] > map

    The code for my main runner class is as follows:-


    public class Es2 {

              static private final Path TMP_DIR = new

Path(Es2.class.getSimpleName()

                  + "_TMP_1");


          /**

           * @param args the command line arguments

           */

          public static void main(String[] args) throws IOException{

         //#String node="192.168.1.202";

          JobConf conf = new JobConf();

          conf.set("es.resource", "data/content");

          conf.set("es.nodes", "192.168.1.202:9200 <

http://192.168.1.202:9200>");

          conf.set("es.query", "?q=s_start:java");

          conf.setInputFormat(__EsInputFormat.class);

          conf.setMapOutputKeyClass(__Text.class);

          conf.setMapOutputValueClass(__MapWritable.class);

          conf.setOutputKeyClass(Text.__class);

          conf.setOutputValueClass(__MapWritable.class);

          conf.setOutputFormat(__TextOutputFormat.class);

          conf.setMapperClass(mapper1.__class);

          final Path outDir = new Path(TMP_DIR, "out");

          FileOutputFormat.__setOutputPath(conf, outDir);

          JobClient.runJob(conf);

          }

    }

    The code for my mapper class is as follows:-

    public class mapper1 extends MapReduceBase implements

              Mapper{

       @Override

       public void map(Object key, Object value, OutputCollector

output,

                          Reporter reporter) throws IOException {

         Text docId = (Text) key;

         MapWritable doc = (MapWritable) value;

         output.collect(docId,doc);

       }

    }

    Kindly guide me on this issue.

    Thanks

    --
    You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/_msgid/elasticsearch/3b110c43-
_faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-
faa8-4a08-acfa-0fac5fd89585%40googlegroups.com>
<https://groups.google.com/d/__msgid/elasticsearch/3b110c43-
__faa8-4a08-acfa-0fac5fd89585%_40googlegroups.com?utm
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-
faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=
email&utm_source=footer>>.
For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to a topic in

the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__
topic/elasticsearch/HS1A___psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_
psM7fw/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@__googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/_msgid/elasticsearch/540ED720.
_5040501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540ED720.
5040501%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%
2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%
2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%
40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/HS1A_psM7fw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/540EFF05.6040508%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBKTUr11ck6rSvbzZuf%2Bt1C_tFMiZP407kvrDK4Yfd5P%3DQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

If by error you mean the ClassNotFoundException, you need to check again your classpath. Also be sure to add es-hadoop
to your job classpath (typically pack it with the jar) - the documentation
describes some of the options available [1]

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/mapreduce.html#_installation

On 9/9/14 10:26 PM, gaurav redkar wrote:

Hi Costin,

I had downloaded the elasticsearch-hadoop-2.1.0.Beta1.zip file and used all the jars from that for the program. Later I
even tried replacing all the jars in my program with jars from with elasticsearch-hadoop-2.0.0.zip file, but still
facing the same error.

On Tue, Sep 9, 2014 at 6:52 PM, Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com> wrote:

Most likely you have a classpath conflict caused by multiple versions of es-hadoop. Can you double check you only
have one version (2.1.0.Beta1) available?
Based on the error, I'm guessing you have some 1.3 Mx or the RC somewhere in there...

On 9/9/14 4:06 PM, gaurav redkar wrote:

    Hi Costin,

    Thanks for the heads up regarding gist. I will try to follow the guidelines in the future. As for my program, I
    am using
    Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and changed the output value class to
    LinkedMapWritable. but
    now I am getting the following error.

    https://gist.github.com/__gauravub/7d55bc6b10cb63935eb8 <https://gist.github.com/gauravub/7d55bc6b10cb63935eb8>

    Any idea why is this happening ? I even tried using the v2.0.0 of es-hadoop but am still getting the same error.

    On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__> wrote:

         Hi,

         What version of es-hadoop are you using? The problem stems from the difference in the types mentioned on your
         Mapper, namely the output value class:

                   conf.setMapOutputValueClass(____MapWritable.class);


         to MapWritable while LinkedMapWritable is returned. The latest versions automatically detect this and use
    the proper
         type so I recommend upgrading.
         If that's not an option, use LinkedMapWritable.

         Cheers,

         P.S. Please don't post code and stracktraces on the mailing list since it highly reduces the readability of
    your
         email. Instead use gist or any other service
         to post the code as indicated in the docs [1]. Thanks

         [1]
    http://www.elasticsearch.org/____guide/en/elasticsearch/__hadoop/__2.1.Beta/__troubleshooting.html#___where___do_i_post_my___information
    <http://www.elasticsearch.org/__guide/en/elasticsearch/hadoop/__2.1.Beta/troubleshooting.html#___where_do_i_post_my___information>

    <http://www.elasticsearch.org/__guide/en/elasticsearch/hadoop/__2.1.Beta/troubleshooting.html#___where_do_i_post_my___information
    <http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/troubleshooting.html#_where_do_i_post_my_information>>




         On 9/9/14 11:59 AM, gaurav redkar wrote:

             Hi, I was following the example given on official elasticsearch documentation to read data from
    elasticsearch using
             hadoop but i am getting the following error.

             java.lang.Exception: java.io.IOException: Type mismatch in value from map: expected
    org.apache.hadoop.io <http://org.apache.hadoop.io>
             <http://org.apache.hadoop.io>.____MapWritable,
             received org.elasticsearch.hadoop.mr <http://org.elasticsearch.hadoop.mr>
    <http://org.elasticsearch.__hadoop.mr <http://org.elasticsearch.hadoop.mr>>.__LinkedMapWritable

             at org.apache.hadoop.mapred.____LocalJobRunner$Job.runTasks(____LocalJobRunner.java:462)

             at org.apache.hadoop.mapred.____LocalJobRunner$Job.run(____LocalJobRunner.java:522)

             Caused by: java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io
    <http://org.apache.hadoop.io>
             <http://org.apache.hadoop.io>.____MapWritable, received
    org.elasticsearch.hadoop.mr <http://org.elasticsearch.hadoop.mr> <http://org.elasticsearch.__hadoop.mr
    <http://org.elasticsearch.hadoop.mr>>.__LinkedMapWritable

             at org.apache.hadoop.mapred.____MapTask$MapOutputBuffer.____collect(MapTask.java:1054)

             at org.apache.hadoop.mapred.____MapTask$OldOutputCollector.____collect(MapTask.java:587)

             at es2.mapper1.map(mapper1.java:____29)

             at org.apache.hadoop.mapred.____MapRunner.run(MapRunner.java:____54)

             at org.apache.hadoop.mapred.____MapTask.runOldMapper(MapTask.____java:430)

             at org.apache.hadoop.mapred.____MapTask.run(MapTask.java:342)

             at org.apache.hadoop.mapred.____LocalJobRunner$Job$____MapTaskRunnable.run(____LocalJobRunner.java:243)

             at java.util.concurrent.____Executors$RunnableAdapter.____call(Executors.java:471)

             at java.util.concurrent.____FutureTask$Sync.innerRun(____FutureTask.java:334)

             at java.util.concurrent.____FutureTask.run(FutureTask.____java:166)

             at java.util.concurrent.____ThreadPoolExecutor.runWorker(____ThreadPoolExecutor.java:1145)

             at java.util.concurrent.____ThreadPoolExecutor$Worker.run(____ThreadPoolExecutor.java:615)

             at java.lang.Thread.run(Thread.____java:724)

             14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed with state FAILED due to: NA

             14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0

             Exception in thread "main" java.io.IOException: Job failed!

             at org.apache.hadoop.mapred.____JobClient.runJob(JobClient.____java:836)

             at es2.Es2.main(Es2.java:48)

             at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native Method)

             at sun.reflect.____NativeMethodAccessorImpl.____invoke(____NativeMethodAccessorImpl.java:____57)

             at sun.reflect.____DelegatingMethodAccessorImpl.____invoke(____DelegatingMethodAccessorImpl.____java:43)

             at java.lang.reflect.Method.____invoke(Method.java:606)

             at org.apache.hadoop.util.RunJar.____main(RunJar.java:212)

             14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
             [node=[QMYbfSRMTsyXVVgfFweMkA/____Conceptnet|192.168.1.202:__9200 <http://192.168.1.202:9200>
    <http://192.168.1.202:9200>]____,shard=0] > map


             The code for my main runner class is as follows:-


             public class Es2 {

                       static private final Path TMP_DIR = new Path(Es2.class.getSimpleName()

                           + "_TMP_1");


                   /**

                    * @param args the command line arguments

                    */

                   public static void main(String[] args) throws IOException{

                  //#String node="192.168.1.202";

                   JobConf conf = new JobConf();

                   conf.set("es.resource", "data/content");

                   conf.set("es.nodes", "192.168.1.202:9200 <http://192.168.1.202:9200> <http://192.168.1.202:9200>");

                   conf.set("es.query", "?q=s_start:java");

                   conf.setInputFormat(____EsInputFormat.class);

                   conf.setMapOutputKeyClass(____Text.class);

                   conf.setMapOutputValueClass(____MapWritable.class);

                   conf.setOutputKeyClass(Text.____class);

                   conf.setOutputValueClass(____MapWritable.class);

                   conf.setOutputFormat(____TextOutputFormat.class);

                   conf.setMapperClass(mapper1.____class);

                   final Path outDir = new Path(TMP_DIR, "out");

                   FileOutputFormat.____setOutputPath(conf, outDir);

                   JobClient.runJob(conf);

                   }

             }

             The code for my mapper class is as follows:-

             public class mapper1 extends MapReduceBase implements

                       Mapper{

                @Override

                public void map(Object key, Object value, OutputCollector output,

                                   Reporter reporter) throws IOException {

                  Text docId = (Text) key;

                  MapWritable doc = (MapWritable) value;

                  output.collect(docId,doc);

                }

             }

             Kindly guide me on this issue.

             Thanks

             --
             You received this message because you are subscribed to the Google Groups "elasticsearch" group.
             To unsubscribe from this group and stop receiving emails from it, send an email to
             elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch+____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B__unsubscribe@googlegroups.com> <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>>.
             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=email&utm_source=footer>>>.
             For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


         --
         Costin

         --
         You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
         To unsubscribe from this topic, visit
    https://groups.google.com/d/____topic/elasticsearch/HS1A_____psM7fw/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe>
         <https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/HS1A_psM7fw/unsubscribe>>.
         To unsubscribe from this group and all its topics, send an email to
    elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
         <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>.
         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/540ED720.____5040501%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/540ED720.__5040501%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/540ED720.__5040501%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/540ED720.5040501%40gmail.com>>.

         For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%__2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%__40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%__2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%__40mail.gmail.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_psM7fw/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/540EFF05.__6040508%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540EFF05.6040508%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBKTUr11ck6rSvbzZuf%2Bt1C_tFMiZP407kvrDK4Yfd5P%3DQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBKTUr11ck6rSvbzZuf%2Bt1C_tFMiZP407kvrDK4Yfd5P%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/540F61E9.2030303%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Hi Costin,

Thanks for your inputs. I was able to get it running after I copied the
elasticsearch-hadoop-2.0.0.jar to the "lib" directory of my hadoop
installation. The reason why I was stuck with this issue is because i had
already packaged this es-hadoop jar into my application and had built a
jar. So when I was running the example as follows :-

hadoop jar es2.jar Es2

where Es2 is the name of the runner class which the main() function, I was
expecting the program to find the required classes since I had already
bundled the es-hadoop jar within the project jar.

Also in the instructions on the elasticsearch-hadoop documentation at

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/mapreduce.html#CO14-2

it was mentioned to add the jar to the HADOOP_CLASSPATH. I first added the
path to the es-hadoop.jar to the HADOOP_CLASSPATH, but it gave the same
error. Later I added the jar to one of paths mentioned within
HADOOP_CLASSPATH, and the program executed. Can you guide as to why is it
working in the second case and not the first case ? Or am I doing something
wrong ?

Anyway thanks for your guidance.

Regards,
Gaurav

On Wed, Sep 10, 2014 at 1:54 AM, Costin Leau costin.leau@gmail.com wrote:

If by error you mean the ClassNotFoundException, you need to check again
your classpath. Also be sure to add es-hadoop to your job classpath
(typically pack it with the jar) - the documentation
describes some of the options available [1]

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/
2.1.Beta/mapreduce.html#_installation

On 9/9/14 10:26 PM, gaurav redkar wrote:

Hi Costin,

I had downloaded the elasticsearch-hadoop-2.1.0.Beta1.zip file and used
all the jars from that for the program. Later I
even tried replacing all the jars in my program with jars from with
elasticsearch-hadoop-2.0.0.zip file, but still
facing the same error.

On Tue, Sep 9, 2014 at 6:52 PM, Costin Leau <costin.leau@gmail.com
mailto:costin.leau@gmail.com> wrote:

Most likely you have a classpath conflict caused by multiple versions

of es-hadoop. Can you double check you only
have one version (2.1.0.Beta1) available?
Based on the error, I'm guessing you have some 1.3 Mx or the RC
somewhere in there...

On 9/9/14 4:06 PM, gaurav redkar wrote:

    Hi Costin,

    Thanks for the heads up regarding gist. I will try to follow the

guidelines in the future. As for my program, I
am using
Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and
changed the output value class to
LinkedMapWritable. but
now I am getting the following error.

    https://gist.github.com/__gauravub/7d55bc6b10cb63935eb8 <

https://gist.github.com/gauravub/7d55bc6b10cb63935eb8>

    Any idea why is this happening ? I even tried using the v2.0.0 of

es-hadoop but am still getting the same error.

    On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <

costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>__>
wrote:

         Hi,

         What version of es-hadoop are you using? The problem stems

from the difference in the types mentioned on your
Mapper, namely the output value class:

                   conf.setMapOutputValueClass(__

__MapWritable.class);

         to MapWritable while LinkedMapWritable is returned. The

latest versions automatically detect this and use
the proper
type so I recommend upgrading.
If that's not an option, use LinkedMapWritable.

         Cheers,

         P.S. Please don't post code and stracktraces on the mailing

list since it highly reduces the readability of
your
email. Instead use gist or any other service
to post the code as indicated in the docs [1]. Thanks

         [1]
    http://www.elasticsearch.org/____guide/en/elasticsearch/__

hadoop/__2.1.Beta/__troubleshooting.html#__where
__do_i_post_my___information
<http://www.elasticsearch.org/__guide/en/elasticsearch/
hadoop/__2.1.Beta/troubleshooting.html#__where
do_i_post_my___information>

    <http://www.elasticsearch.org/__guide/en/elasticsearch/

hadoop/__2.1.Beta/troubleshooting.html#__where
do_i_post_my___information
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/
2.1.Beta/troubleshooting.html#_where_do_i_post_my_information>>

         On 9/9/14 11:59 AM, gaurav redkar wrote:

             Hi, I was following the example given on official

elasticsearch documentation to read data from
elasticsearch using
hadoop but i am getting the following error.

             java.lang.Exception: java.io.IOException: Type mismatch

in value from map: expected
org.apache.hadoop.io http://org.apache.hadoop.io
http://org.apache.hadoop.io.____MapWritable,
received org.elasticsearch.hadoop.mr <
http://org.elasticsearch.hadoop.mr>
<http://org.elasticsearch.__hadoop.mr <http://org.elasticsearch.
hadoop.mr>>.__LinkedMapWritable

             at org.apache.hadoop.mapred.____

LocalJobRunner$Job.runTasks(____LocalJobRunner.java:462)

             at org.apache.hadoop.mapred.____

LocalJobRunner$Job.run(____LocalJobRunner.java:522)

             Caused by: java.io.IOException: Type mismatch in value

from map: expected org.apache.hadoop.io
http://org.apache.hadoop.io
http://org.apache.hadoop.io.____MapWritable, received
org.elasticsearch.hadoop.mr http://org.elasticsearch.hadoop.mr
<http://org.elasticsearch.__hadoop.mr
http://org.elasticsearch.hadoop.mr>.__LinkedMapWritable

             at org.apache.hadoop.mapred.____

MapTask$MapOutputBuffer.____collect(MapTask.java:1054)

             at org.apache.hadoop.mapred.____

MapTask$OldOutputCollector.____collect(MapTask.java:587)

             at es2.mapper1.map(mapper1.java:____29)

             at org.apache.hadoop.mapred.____

MapRunner.run(MapRunner.java:____54)

             at org.apache.hadoop.mapred.____

MapTask.runOldMapper(MapTask.____java:430)

             at org.apache.hadoop.mapred.____

MapTask.run(MapTask.java:342)

             at org.apache.hadoop.mapred.____LocalJobRunner$Job$____

MapTaskRunnable.run(____LocalJobRunner.java:243)

             at java.util.concurrent.____

Executors$RunnableAdapter.____call(Executors.java:471)

             at java.util.concurrent.____

FutureTask$Sync.innerRun(____FutureTask.java:334)

             at java.util.concurrent.____

FutureTask.run(FutureTask.____java:166)

             at java.util.concurrent.____

ThreadPoolExecutor.runWorker(____ThreadPoolExecutor.java:1145)

             at java.util.concurrent.____

ThreadPoolExecutor$Worker.run(____ThreadPoolExecutor.java:615)

             at java.lang.Thread.run(Thread.____java:724)

             14/09/09 13:25:51 INFO mapreduce.Job: Job

job_local994189535_0001 failed with state FAILED due to: NA

             14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0

             Exception in thread "main" java.io.IOException: Job

failed!

             at org.apache.hadoop.mapred.____

JobClient.runJob(JobClient.____java:836)

             at es2.Es2.main(Es2.java:48)

             at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native

Method)

             at sun.reflect.____NativeMethodAccessorImpl.____

invoke(____NativeMethodAccessorImpl.java:____57)

             at sun.reflect.____DelegatingMethodAccessorImpl._

___invoke(____DelegatingMethodAccessorImpl.____java:43)

             at java.lang.reflect.Method.____invoke(Method.java:606)

             at org.apache.hadoop.util.RunJar.

____main(RunJar.java:212)

             14/09/09 13:25:51 INFO mapred.LocalJobRunner:

ShardInputSplit
[node=[QMYbfSRMTsyXVVgfFweMkA/
Conceptnet|192.168.1.202:__9200 http://192.168.1.202:9200
http://192.168.1.202:9200]
,shard=0] > map

             The code for my main runner class is as follows:-


             public class Es2 {

                       static private final Path TMP_DIR = new

Path(Es2.class.getSimpleName()

                           + "_TMP_1");


                   /**

                    * @param args the command line arguments

                    */

                   public static void main(String[] args) throws

IOException{

                  //#String node="192.168.1.202";

                   JobConf conf = new JobConf();

                   conf.set("es.resource", "data/content");

                   conf.set("es.nodes", "192.168.1.202:9200 <

http://192.168.1.202:9200> http://192.168.1.202:9200");

                   conf.set("es.query", "?q=s_start:java");

                   conf.setInputFormat(____EsInputFormat.class);

                   conf.setMapOutputKeyClass(____Text.class);

                   conf.setMapOutputValueClass(__

__MapWritable.class);

                   conf.setOutputKeyClass(Text.____class);

                   conf.setOutputValueClass(____MapWritable.class);

                   conf.setOutputFormat(____TextOutputFormat.class);

                   conf.setMapperClass(mapper1.____class);

                   final Path outDir = new Path(TMP_DIR, "out");

                   FileOutputFormat.____setOutputPath(conf, outDir);

                   JobClient.runJob(conf);

                   }

             }

             The code for my mapper class is as follows:-

             public class mapper1 extends MapReduceBase implements

                       Mapper{

                @Override

                public void map(Object key, Object value,

OutputCollector output,

                                   Reporter reporter) throws

IOException {

                  Text docId = (Text) key;

                  MapWritable doc = (MapWritable) value;

                  output.collect(docId,doc);

                }

             }

             Kindly guide me on this issue.

             Thanks

             --
             You received this message because you are subscribed to

the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>
>
<mailto:elasticsearch+____unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com
>>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/3b110c43-
__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-

__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-
faa8-4a08-acfa-0fac5fd89585%40googlegroups.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/

3b110c43-____faa8-4a08-acfa-0fac5fd89585%_40googlegroups.com?utm
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/3b110c43-
__faa8-4a08-acfa-0fac5fd89585%_40googlegroups.com?utm
medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-

__faa8-4a08-acfa-0fac5fd89585%_40googlegroups.com?utm
medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-
faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=
email&utm_source=footer>>>.
For more options, visit https://groups.google.com/d/

__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.

         --
         Costin

         --
         You received this message because you are subscribed to a

topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/HS1A
psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A

psM7fw/unsubscribe>
<https://groups.google.com/d/topic/elasticsearch/HS1A_
psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_
psM7fw/unsubscribe>>.
To unsubscribe from this group and all its topics, send an
email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com
>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
540ED720.____5040501%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/540ED720.
__5040501%40gmail.com>
<https://groups.google.com/d/__msgid/elasticsearch/540ED720.
__5040501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540ED720.
5040501%40gmail.com>>.

         For more options, visit https://groups.google.com/d/__

__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.

    --
    You received this message because you are subscribed to the

Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/_msgid/elasticsearch/CAJ1vTBJ%
_2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%__40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%
2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com>
<https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%
2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%
2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%
40mail.gmail.com?utm_medium=email&utm_source=footer>>.
For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to a topic in

the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__
topic/elasticsearch/HS1A___psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_
psM7fw/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@__googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/_msgid/elasticsearch/540EFF05.
_6040508%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540EFF05.
6040508%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <

https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBKTUr11ck6rSvbzZuf%
2Bt1C_tFMiZP407kvrDK4Yfd5P%3DQ%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CAJ1vTBKTUr11ck6rSvbzZuf%2Bt1C_tFMiZP407kvrDK4Yfd5P%
3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/HS1A_psM7fw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/540F61E9.2030303%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJze%2BfmPRz52i03pmAT1F29CPA_mUzLtfZKjN2mkaSf1Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Unfortunately there's no clear answer since each distro and version (in particular 1.x vs 2.x) added more options to
configure the classpath and thus configure the cluster and its jobs.
This is actually the reason why in the docs we don't recommend one big way - the classpath config is fragile (the order
of the arguments matter, one settings can override another, a different env variable is read instead of another) etc...

You could try following up with the distro (apache or not) docs and potentially mailing list. Additionally you can try
checking out the sources in particular the start-up scripts...

On 9/11/14 9:13 AM, gaurav redkar wrote:

Hi Costin,

Thanks for your inputs. I was able to get it running after I copied the elasticsearch-hadoop-2.0.0.jar to the "lib"
directory of my hadoop installation. The reason why I was stuck with this issue is because i had already packaged this
es-hadoop jar into my application and had built a jar. So when I was running the example as follows :-

hadoop jar es2.jar Es2

where Es2 is the name of the runner class which the main() function, I was expecting the program to find the required
classes since I had already bundled the es-hadoop jar within the project jar.

Also in the instructions on the elasticsearch-hadoop documentation at

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/mapreduce.html#CO14-2

it was mentioned to add the jar to the HADOOP_CLASSPATH. I first added the path to the es-hadoop.jar to the
HADOOP_CLASSPATH, but it gave the same error. Later I added the jar to one of paths mentioned within HADOOP_CLASSPATH,
and the program executed. Can you guide as to why is it working in the second case and not the first case ? Or am I
doing something wrong ?

Anyway thanks for your guidance.

Regards,
Gaurav

On Wed, Sep 10, 2014 at 1:54 AM, Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com> wrote:

If by error you mean the ClassNotFoundException, you need to check again your classpath. Also be sure to add
es-hadoop to your job classpath (typically pack it with the jar) - the documentation
describes some of the options available [1]

[1] http://www.elasticsearch.org/__guide/en/elasticsearch/hadoop/__2.1.Beta/mapreduce.html#___installation
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/mapreduce.html#_installation>


On 9/9/14 10:26 PM, gaurav redkar wrote:

    Hi Costin,

    I had downloaded the elasticsearch-hadoop-2.1.0.__Beta1.zip file and used all the jars from that for the
    program. Later I
    even  tried replacing all the jars in my program with jars from with elasticsearch-hadoop-2.0.0.zip file, but still
    facing the same error.

    On Tue, Sep 9, 2014 at 6:52 PM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__> wrote:

         Most likely you have a classpath conflict caused by multiple versions of es-hadoop. Can you double check
    you only
         have one version (2.1.0.Beta1) available?
         Based on the error, I'm guessing you have some 1.3 Mx or the RC somewhere in there...

         On 9/9/14 4:06 PM, gaurav redkar wrote:

             Hi Costin,

             Thanks for the heads up regarding gist. I will try to follow the guidelines in the future. As for my
    program, I
             am using
             Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and changed the output value class to
             LinkedMapWritable. but
             now I am getting the following error.

    https://gist.github.com/____gauravub/7d55bc6b10cb63935eb8
    <https://gist.github.com/__gauravub/7d55bc6b10cb63935eb8>
    <https://gist.github.com/__gauravub/7d55bc6b10cb63935eb8 <https://gist.github.com/gauravub/7d55bc6b10cb63935eb8>>

             Any idea why is this happening ? I even tried using the v2.0.0 of es-hadoop but am still getting the
    same error.

             On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>>__>__> wrote:

                  Hi,

                  What version of es-hadoop are you using? The problem stems from the difference in the types
    mentioned on your
                  Mapper, namely the output value class:

                            conf.setMapOutputValueClass(______MapWritable.class);


                  to MapWritable while LinkedMapWritable is returned. The latest versions automatically detect this
    and use
             the proper
                  type so I recommend upgrading.
                  If that's not an option, use LinkedMapWritable.

                  Cheers,

                  P.S. Please don't post code and stracktraces on the mailing list since it highly reduces the
    readability of
             your
                  email. Instead use gist or any other service
                  to post the code as indicated in the docs [1]. Thanks

                  [1]
    http://www.elasticsearch.org/______guide/en/elasticsearch/____hadoop/__2.1.Beta/____troubleshooting.html#___where_____do_i_post_my___information
    <http://www.elasticsearch.org/____guide/en/elasticsearch/__hadoop/__2.1.Beta/__troubleshooting.html#___where___do_i_post_my___information>

    <http://www.elasticsearch.org/____guide/en/elasticsearch/__hadoop/__2.1.Beta/__troubleshooting.html#___where___do_i_post_my___information
    <http://www.elasticsearch.org/__guide/en/elasticsearch/hadoop/__2.1.Beta/troubleshooting.html#___where_do_i_post_my___information>>


    <http://www.elasticsearch.org/____guide/en/elasticsearch/__hadoop/__2.1.Beta/__troubleshooting.html#___where___do_i_post_my___information
    <http://www.elasticsearch.org/__guide/en/elasticsearch/hadoop/__2.1.Beta/troubleshooting.html#___where_do_i_post_my___information>

    <http://www.elasticsearch.org/__guide/en/elasticsearch/hadoop/__2.1.Beta/troubleshooting.html#___where_do_i_post_my___information
    <http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/troubleshooting.html#_where_do_i_post_my_information>>>




                  On 9/9/14 11:59 AM, gaurav redkar wrote:

                      Hi, I was following the example given on official elasticsearch documentation to read data from
             elasticsearch using
                      hadoop but i am getting the following error.

                      java.lang.Exception: java.io.IOException: Type mismatch in value from map: expected
    org.apache.hadoop.io <http://org.apache.hadoop.io> <http://org.apache.hadoop.io>
                      <http://org.apache.hadoop.io>.______MapWritable,
                      received org.elasticsearch.hadoop.mr <http://org.elasticsearch.hadoop.mr>
    <http://org.elasticsearch.__hadoop.mr <http://org.elasticsearch.hadoop.mr>>
             <http://org.elasticsearch.__ha__doop.mr <http://hadoop.mr> <http://org.elasticsearch.__hadoop.mr
    <http://org.elasticsearch.hadoop.mr>>>.____LinkedMapWritable

                      at org.apache.hadoop.mapred.______LocalJobRunner$Job.runTasks(______LocalJobRunner.java:462)

                      at org.apache.hadoop.mapred.______LocalJobRunner$Job.run(______LocalJobRunner.java:522)

                      Caused by: java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io
    <http://org.apache.hadoop.io>
             <http://org.apache.hadoop.io>
                      <http://org.apache.hadoop.io>.______MapWritable, received
    org.elasticsearch.hadoop.mr <http://org.elasticsearch.hadoop.mr> <http://org.elasticsearch.__hadoop.mr
    <http://org.elasticsearch.hadoop.mr>> <http://org.elasticsearch.__ha__doop.mr <http://hadoop.mr>
             <http://org.elasticsearch.__hadoop.mr <http://org.elasticsearch.hadoop.mr>>>.____LinkedMapWritable

                      at org.apache.hadoop.mapred.______MapTask$MapOutputBuffer.______collect(MapTask.java:1054)

                      at org.apache.hadoop.mapred.______MapTask$OldOutputCollector.______collect(MapTask.java:587)

                      at es2.mapper1.map(mapper1.java:______29)

                      at org.apache.hadoop.mapred.______MapRunner.run(MapRunner.java:______54)

                      at org.apache.hadoop.mapred.______MapTask.runOldMapper(MapTask.______java:430)

                      at org.apache.hadoop.mapred.______MapTask.run(MapTask.java:342)

                      at
    org.apache.hadoop.mapred.______LocalJobRunner$Job$______MapTaskRunnable.run(______LocalJobRunner.java:243)

                      at java.util.concurrent.______Executors$RunnableAdapter.______call(Executors.java:471)

                      at java.util.concurrent.______FutureTask$Sync.innerRun(______FutureTask.java:334)

                      at java.util.concurrent.______FutureTask.run(FutureTask.______java:166)

                      at java.util.concurrent.______ThreadPoolExecutor.runWorker(______ThreadPoolExecutor.java:__1145)

                      at java.util.concurrent.______ThreadPoolExecutor$Worker.run(______ThreadPoolExecutor.java:__615)

                      at java.lang.Thread.run(Thread.______java:724)

                      14/09/09 13:25:51 INFO mapreduce.Job: Job job_local994189535_0001 failed with state FAILED due
    to: NA

                      14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0

                      Exception in thread "main" java.io.IOException: Job failed!

                      at org.apache.hadoop.mapred.______JobClient.runJob(JobClient.______java:836)

                      at es2.Es2.main(Es2.java:48)

                      at sun.reflect.______NativeMethodAccessorImpl.______invoke0(Native Method)

                      at
    sun.reflect.______NativeMethodAccessorImpl.______invoke(______NativeMethodAccessorImpl.java:______57)

                      at
    sun.reflect.______DelegatingMethodAccessorImpl.______invoke(______DelegatingMethodAccessorImpl.______java:43)

                      at java.lang.reflect.Method.______invoke(Method.java:606)

                      at org.apache.hadoop.util.RunJar.______main(RunJar.java:212)

                      14/09/09 13:25:51 INFO mapred.LocalJobRunner: ShardInputSplit
                      [node=[QMYbfSRMTsyXVVgfFweMkA/______Conceptnet|192.168.1.202:____9200 <http://192.168.1.202:9200>
             <http://192.168.1.202:9200>]______,shard=0] > map


                      The code for my main runner class is as follows:-


                      public class Es2 {

                                static private final Path TMP_DIR = new Path(Es2.class.getSimpleName()

                                    + "_TMP_1");


                            /**

                             * @param args the command line arguments

                             */

                            public static void main(String[] args) throws IOException{

                           //#String node="192.168.1.202";

                            JobConf conf = new JobConf();

                            conf.set("es.resource", "data/content");

                            conf.set("es.nodes", "192.168.1.202:9200 <http://192.168.1.202:9200>
    <http://192.168.1.202:9200> <http://192.168.1.202:9200>");

                            conf.set("es.query", "?q=s_start:java");

                            conf.setInputFormat(______EsInputFormat.class);

                            conf.setMapOutputKeyClass(______Text.class);

                            conf.setMapOutputValueClass(______MapWritable.class);

                            conf.setOutputKeyClass(Text.______class);

                            conf.setOutputValueClass(______MapWritable.class);

                            conf.setOutputFormat(______TextOutputFormat.class);

                            conf.setMapperClass(mapper1.______class);

                            final Path outDir = new Path(TMP_DIR, "out");

                            FileOutputFormat.______setOutputPath(conf, outDir);

                            JobClient.runJob(conf);

                            }

                      }

                      The code for my mapper class is as follows:-

                      public class mapper1 extends MapReduceBase implements

                                Mapper{

                         @Override

                         public void map(Object key, Object value, OutputCollector output,

                                            Reporter reporter) throws IOException {

                           Text docId = (Text) key;

                           MapWritable doc = (MapWritable) value;

                           output.collect(docId,doc);

                         }

                      }

                      Kindly guide me on this issue.

                      Thanks

                      --
                      You received this message because you are subscribed to the Google Groups "elasticsearch" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to
                      elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com>
    <http://googlegroups.com>
             <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>
                      <mailto:elasticsearch+______unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B____unsubscribe@googlegroups.com>
             <mailto:elasticsearch%2B____unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B__unsubscribe@googlegroups.com>>
    <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>>.
                      To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__3b110c43-____faa8-4a08-acfa-__0fac5fd89585%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com>>


    <https://groups.google.com/d/____msgid/elasticsearch/3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com>>>


    <https://groups.google.com/d/______msgid/elasticsearch/__3b110c43-____faa8-4a08-acfa-__0fac5fd89585%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com?utm___medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com?utm_medium=__email&utm_source=footer>>


    <https://groups.google.com/d/____msgid/elasticsearch/3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/3b110c43-__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/3b110c43-faa8-4a08-acfa-0fac5fd89585%40googlegroups.com?utm_medium=email&utm_source=footer>>>>.
                      For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.


                  --
                  Costin

                  --
                  You received this message because you are subscribed to a topic in the Google Groups
    "elasticsearch" group.
                  To unsubscribe from this topic, visit
    https://groups.google.com/d/______topic/elasticsearch/HS1A_______psM7fw/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/HS1A_____psM7fw/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/HS1A_____psM7fw/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe>>
                  <https://groups.google.com/d/____topic/elasticsearch/HS1A_____psM7fw/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe>
             <https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/HS1A_psM7fw/unsubscribe>>>.
                  To unsubscribe from this group and all its topics, send an email to
             elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com> <http://googlegroups.com>
                  <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>.
                  To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__540ED720.____5040501%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/540ED720.____5040501%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/540ED720.____5040501%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/540ED720.__5040501%40gmail.com>>
                  <https://groups.google.com/d/____msgid/elasticsearch/540ED720.____5040501%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/540ED720.__5040501%40gmail.com>
             <https://groups.google.com/d/__msgid/elasticsearch/540ED720.__5040501%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/540ED720.5040501%40gmail.com>>>.

                  For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.


             --
             You received this message because you are subscribed to the Google Groups "elasticsearch" group.
             To unsubscribe from this group and stop receiving emails from it, send an email to
             elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch+____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B__unsubscribe@googlegroups.com> <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>>.
             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/CAJ1vTBJ%____2BgBi6oUwbfxJSrvjXWpu_____FzhJ4T3BbSQ%2BNA-iE_NtHA%____40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%__2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%__40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%__2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%__40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/CAJ1vTBJ%____2BgBi6oUwbfxJSrvjXWpu_____FzhJ4T3BbSQ%2BNA-iE_NtHA%____40mail.gmail.com?utm_medium=____email&utm_source=footer
    <https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%__2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%__40mail.gmail.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%__2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%__40mail.gmail.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
             For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.

         --
         Costin

         --
         You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
         To unsubscribe from this topic, visit
    https://groups.google.com/d/____topic/elasticsearch/HS1A_____psM7fw/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe>
         <https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/HS1A_psM7fw/unsubscribe>>.
         To unsubscribe from this group and all its topics, send an email to
    elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
         <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>.
         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/540EFF05.____6040508%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/540EFF05.__6040508%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/540EFF05.__6040508%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/540EFF05.6040508%40gmail.com>>.

         For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/__CAJ1vTBKTUr11ck6rSvbzZuf%__2Bt1C_tFMiZP407kvrDK4Yfd5P%__3DQ%40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBKTUr11ck6rSvbzZuf%2Bt1C_tFMiZP407kvrDK4Yfd5P%3DQ%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__CAJ1vTBKTUr11ck6rSvbzZuf%__2Bt1C_tFMiZP407kvrDK4Yfd5P%__3DQ%40mail.gmail.com?utm___medium=email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBKTUr11ck6rSvbzZuf%2Bt1C_tFMiZP407kvrDK4Yfd5P%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer>__>.
    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__topic/elasticsearch/HS1A___psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_psM7fw/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/540F61E9.__2030303%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540F61E9.2030303%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJze%2BfmPRz52i03pmAT1F29CPA_mUzLtfZKjN2mkaSf1Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJze%2BfmPRz52i03pmAT1F29CPA_mUzLtfZKjN2mkaSf1Q%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/54114E14.6070207%40gmail.com.
For more options, visit https://groups.google.com/d/optout.