Quoting from the same docs again:
"EsOutputFormat expects a Map<Writable, Writable> representing a document
value that is converted interally into a JSON document and indexed in
Elasticsearch. Hadoop OutputFormat requires implementations to expect a key
and a value however, since for Elasticsearch only the document (that is the
value) is necessary, EsOutputFormat ignores the key."
As for MapWritable, it's Hadoop Map implementation - other than that, it's
just a map. It's not something es-hadoop provides but rather Hadoop. I
recommend spending some time getting familiar with Hadoop since es-hadoop
leverages many of its classes and concepts; after all it's a dedicated
'connector' to Elasticsearch for Hadoop.
P.S. In the future, please don't hijack old threads but rather start your
own - it's easier for everyone.
On Sat, Jul 26, 2014 at 12:19 AM, M_20 rastegar.83@gmail.com wrote:
Costin,
Thank you for your reply. I've read the official docs before. But it seems
I am missing something. So, I wanted to see a complete example to make sure
I am understanding ES-hadoop correctly.
For example, in the official docs, it talks about writing data to ES, and
this is the mapper
/////////////////////////////////////
public void map(Object key, Object value, OutputCollector output,
Reporter reporter) throws IOException {
// create the MapWritable object
MapWritable doc = new MapWritable();
...
// write the result to the output collector
// one can pass whatever value to the key; EsOutputFormat ignores it
output.collect(NullWritable.get(), map);
}}
/////////////////////////////////////
Could you please tell me what's map in
output.collect(NullWritable.get(), map);
Also, could you please tell me a little bit about "MapWritable doc"? How its format should be?
Regards
On Friday, July 25, 2014 3:25:21 PM UTC-5, Costin Leau wrote:
M_20, I've already replied to your initial query on where you can find
some example - the official docs:
Elasticsearch Platform — Find real-time answers at scale | Elastic
current/mapreduce.html
There's also google which points to other resources outside Elasticsearch.
On Fri, Jul 25, 2014 at 11:10 PM, M_20 raste...@gmail.com wrote:
Hi Chetana,
Could you please share with me a java sample code of Map/reduce on
Elastcisearch-Hadoop?
Regards
On Thursday, April 3, 2014 4:36:24 AM UTC-5, Chetana wrote:
I have downloaded elasticsearch-hadoop-1.2.0.jar from github and trying
to search. The code to search looks like
Configuration conf = new Configuration();
conf.set("es.nodes", "localhost");
conf.set("es.port", "9200");
conf.set("es.resource", "test_index/test_mapping");
conf.set("es.query", "{"query":{"term":{"test.
field1":"test"}}}");
Job job = Job.getInstance(conf);
job.setJobName(ESIndexContoller.class.getSimpleName());
job.setJarByClass(ESIndexContoller.class);
job.setInputFormatClass(EsInputFormat.class);
job.setOutputFormatClass(EsOutputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(MapWritable.class);
job.setSpeculativeExecution(false);
job.waitForCompletion(true);
I have hadoop 2.2 from hortonworks and ES 0.90.2. But when I run
'hadoop jar' command it throws 'IllegalStateException: Job in state DEFINE
instead of RUNNING'
I believe one needs to create custom Mapper and Reducer. If so, can
someone tell me how the code should look like inside map() and reduce()
Thanks,
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/c54ca61b-26d8-4cad-91ee-80552ae13116%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c54ca61b-26d8-4cad-91ee-80552ae13116%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/377ea0b8-ea98-4a38-9f83-a58849135667%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/377ea0b8-ea98-4a38-9f83-a58849135667%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJogdmd-4CZSPdrRxr%3DS_J7SgvUP7%2BusqeN0XMASh18CBpc2Mw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.