Hi Costin,
I had downloaded the elasticsearch-hadoop-2.1.0.Beta1.zip file and used
all the jars from that for the program. Later I
even tried replacing all the jars in my program with jars from with
elasticsearch-hadoop-2.0.0.zip file, but still
facing the same error.
On Tue, Sep 9, 2014 at 6:52 PM, Costin Leau <costin.leau@gmail.com
mailto:costin.leau@gmail.com> wrote:
Most likely you have a classpath conflict caused by multiple versions
of es-hadoop. Can you double check you only
have one version (2.1.0.Beta1) available?
Based on the error, I'm guessing you have some 1.3 Mx or the RC
somewhere in there...
On 9/9/14 4:06 PM, gaurav redkar wrote:
Hi Costin,
Thanks for the heads up regarding gist. I will try to follow the
guidelines in the future. As for my program, I
am using
Elasticsearch Hadoop v2.1.0.Beta1 . I tried your suggestion and
changed the output value class to
LinkedMapWritable. but
now I am getting the following error.
https://gist.github.com/__gauravub/7d55bc6b10cb63935eb8 <
error while reading data from elasticsearch in hadoop · GitHub>
Any idea why is this happening ? I even tried using the v2.0.0 of
es-hadoop but am still getting the same error.
On Tue, Sep 9, 2014 at 4:02 PM, Costin Leau <
costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>__>
wrote:
Hi,
What version of es-hadoop are you using? The problem stems
from the difference in the types mentioned on your
Mapper, namely the output value class:
conf.setMapOutputValueClass(__
__MapWritable.class);
to MapWritable while LinkedMapWritable is returned. The
latest versions automatically detect this and use
the proper
type so I recommend upgrading.
If that's not an option, use LinkedMapWritable.
Cheers,
P.S. Please don't post code and stracktraces on the mailing
list since it highly reduces the readability of
your
email. Instead use gist or any other service
to post the code as indicated in the docs [1]. Thanks
[1]
http://www.elasticsearch.org/____guide/en/elasticsearch/__
hadoop/__2.1.Beta/__troubleshooting.html#__where
__do_i_post_my___information
<Elasticsearch Platform — Find real-time answers at scale | Elastic
hadoop/__2.1.Beta/troubleshooting.html#__where
do_i_post_my___information>
<http://www.elasticsearch.org/__guide/en/elasticsearch/
hadoop/__2.1.Beta/troubleshooting.html#__where
do_i_post_my___information
<Elasticsearch Platform — Find real-time answers at scale | Elastic
2.1.Beta/troubleshooting.html#_where_do_i_post_my_information>>
On 9/9/14 11:59 AM, gaurav redkar wrote:
Hi, I was following the example given on official
elasticsearch documentation to read data from
elasticsearch using
hadoop but i am getting the following error.
java.lang.Exception: java.io.IOException: Type mismatch
in value from map: expected
org.apache.hadoop.io http://org.apache.hadoop.io
http://org.apache.hadoop.io.____MapWritable,
received org.elasticsearch.hadoop.mr <
http://org.elasticsearch.hadoop.mr>
<http://org.elasticsearch.__hadoop.mr <http://org.elasticsearch.
hadoop.mr>>.__LinkedMapWritable
at org.apache.hadoop.mapred.____
LocalJobRunner$Job.runTasks(____LocalJobRunner.java:462)
at org.apache.hadoop.mapred.____
LocalJobRunner$Job.run(____LocalJobRunner.java:522)
Caused by: java.io.IOException: Type mismatch in value
from map: expected org.apache.hadoop.io
http://org.apache.hadoop.io
http://org.apache.hadoop.io.____MapWritable, received
org.elasticsearch.hadoop.mr http://org.elasticsearch.hadoop.mr
<http://org.elasticsearch.__hadoop.mr
http://org.elasticsearch.hadoop.mr>.__LinkedMapWritable
at org.apache.hadoop.mapred.____
MapTask$MapOutputBuffer.____collect(MapTask.java:1054)
at org.apache.hadoop.mapred.____
MapTask$OldOutputCollector.____collect(MapTask.java:587)
at es2.mapper1.map(mapper1.java:____29)
at org.apache.hadoop.mapred.____
MapRunner.run(MapRunner.java:____54)
at org.apache.hadoop.mapred.____
MapTask.runOldMapper(MapTask.____java:430)
at org.apache.hadoop.mapred.____
MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.____LocalJobRunner$Job$____
MapTaskRunnable.run(____LocalJobRunner.java:243)
at java.util.concurrent.____
Executors$RunnableAdapter.____call(Executors.java:471)
at java.util.concurrent.____
FutureTask$Sync.innerRun(____FutureTask.java:334)
at java.util.concurrent.____
FutureTask.run(FutureTask.____java:166)
at java.util.concurrent.____
ThreadPoolExecutor.runWorker(____ThreadPoolExecutor.java:1145)
at java.util.concurrent.____
ThreadPoolExecutor$Worker.run(____ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.____java:724)
14/09/09 13:25:51 INFO mapreduce.Job: Job
job_local994189535_0001 failed with state FAILED due to: NA
14/09/09 13:25:51 INFO mapreduce.Job: Counters: 0
Exception in thread "main" java.io.IOException: Job
failed!
at org.apache.hadoop.mapred.____
JobClient.runJob(JobClient.____java:836)
at es2.Es2.main(Es2.java:48)
at sun.reflect.____NativeMethodAccessorImpl.____invoke0(Native
Method)
at sun.reflect.____NativeMethodAccessorImpl.____
invoke(____NativeMethodAccessorImpl.java:____57)
at sun.reflect.____DelegatingMethodAccessorImpl._
___invoke(____DelegatingMethodAccessorImpl.____java:43)
at java.lang.reflect.Method.____invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.
____main(RunJar.java:212)
14/09/09 13:25:51 INFO mapred.LocalJobRunner:
ShardInputSplit
[node=[QMYbfSRMTsyXVVgfFweMkA/
Conceptnet|192.168.1.202:__9200 http://192.168.1.202:9200
http://192.168.1.202:9200],shard=0] > map
The code for my main runner class is as follows:-
public class Es2 {
static private final Path TMP_DIR = new
Path(Es2.class.getSimpleName()
+ "_TMP_1");
/**
* @param args the command line arguments
*/
public static void main(String[] args) throws
IOException{
//#String node="192.168.1.202";
JobConf conf = new JobConf();
conf.set("es.resource", "data/content");
conf.set("es.nodes", "192.168.1.202:9200 <
http://192.168.1.202:9200> http://192.168.1.202:9200");
conf.set("es.query", "?q=s_start:java");
conf.setInputFormat(____EsInputFormat.class);
conf.setMapOutputKeyClass(____Text.class);
conf.setMapOutputValueClass(__
__MapWritable.class);
conf.setOutputKeyClass(Text.____class);
conf.setOutputValueClass(____MapWritable.class);
conf.setOutputFormat(____TextOutputFormat.class);
conf.setMapperClass(mapper1.____class);
final Path outDir = new Path(TMP_DIR, "out");
FileOutputFormat.____setOutputPath(conf, outDir);
JobClient.runJob(conf);
}
}
The code for my mapper class is as follows:-
public class mapper1 extends MapReduceBase implements
Mapper{
@Override
public void map(Object key, Object value,
OutputCollector output,
Reporter reporter) throws
IOException {
Text docId = (Text) key;
MapWritable doc = (MapWritable) value;
output.collect(docId,doc);
}
}
Kindly guide me on this issue.
Thanks
--
You received this message because you are subscribed to
the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>>
<mailto:elasticsearch+____unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
3b110c43-____faa8-4a08-acfa-0fac5fd89585%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/3b110c43-
__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com>
<https://groups.google.com/d/__msgid/elasticsearch/3b110c43-
__faa8-4a08-acfa-0fac5fd89585%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-
faa8-4a08-acfa-0fac5fd89585%40googlegroups.com>>
<https://groups.google.com/d/____msgid/elasticsearch/
3b110c43-____faa8-4a08-acfa-0fac5fd89585%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/3b110c43-
__faa8-4a08-acfa-0fac5fd89585%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer>
<https://groups.google.com/d/__msgid/elasticsearch/3b110c43-
__faa8-4a08-acfa-0fac5fd89585%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/3b110c43-
faa8-4a08-acfa-0fac5fd89585%40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
email&utm_source=footer>>>.
For more options, visit https://groups.google.com/d/__
__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.
--
Costin
--
You received this message because you are subscribed to a
topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/____topic/elasticsearch/HS1A____
_psM7fw/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/HS1A___
psM7fw/unsubscribe>
<https://groups.google.com/d/__topic/elasticsearch/HS1A___
psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_
psM7fw/unsubscribe>>.
To unsubscribe from this group and all its topics, send an
email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
540ED720.____5040501%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/540ED720.
__5040501%40gmail.com>
<https://groups.google.com/d/__msgid/elasticsearch/540ED720.
__5040501%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540ED720.
5040501%40gmail.com>>.
For more options, visit https://groups.google.com/d/__
__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.
--
You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%_
_2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%__40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%
2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%40mail.gmail.com>
<https://groups.google.com/d/__msgid/elasticsearch/CAJ1vTBJ%
2BgBi6oUwbfxJSrvjXWpu___FzhJ4T3BbSQ%2BNA-iE_NtHA%
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBJ%
2BgBi6oUwbfxJSrvjXWpu_FzhJ4T3BbSQ%2BNA-iE_NtHA%
40mail.gmail.com?utm_medium=email&utm_source=footer>>.
For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.
--
Costin
--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__
topic/elasticsearch/HS1A___psM7fw/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/HS1A_
psM7fw/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@__googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/540EFF05._
_6040508%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/540EFF05.
6040508%40gmail.com>.
For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJ1vTBKTUr11ck6rSvbzZuf%
2Bt1C_tFMiZP407kvrDK4Yfd5P%3DQ%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CAJ1vTBKTUr11ck6rSvbzZuf%2Bt1C_tFMiZP407kvrDK4Yfd5P%
3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.