# Mapreduce json indexing

**URL:** <https://discuss.elastic.co/t/mapreduce-json-indexing/49778>\
**Category:** Elasticsearch\
**Created:** [May 11, 2016, 2:28pm UTC](https://discuss.elastic.co/t/mapreduce-json-indexing/49778 "2016-05-11T14:28:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![fede](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fede/32/126283_2.png) [@fede](https://discuss.elastic.co/u/fede)\
**Post date:** [May 11, 2016, 2:28pm UTC](https://discuss.elastic.co/t/mapreduce-json-indexing/49778/1 "2016-05-11T14:28:27Z")

</div>

I'm trying to index some data using elasticsearch-mapreduce 2.3.1, my input are Avro files that I then convert to JSON documents and I'm getting this error:  
Error: org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: Found unrecoverable error [130.226.238.240:9200] returned Bad Request(400) - failed to parse;mapper [values] of different type, current\_type [long], merged\_type [string]; Bailing out..  
at org.elasticsearch.hadoop.rest.RestClient.retryFailedEntries(RestClient.java:207)  
at org.elasticsearch.hadoop.rest.RestClient.bulk(RestClient.java:170)  
at org.elasticsearch.hadoop.rest.RestRepository.tryFlush(RestRepository.java:225)  
at org.elasticsearch.hadoop.rest.RestRepository.flush(RestRepository.java:248)  
at org.elasticsearch.hadoop.rest.RestRepository.close(RestRepository.java:267)  
at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.doClose(EsOutputFormat.java:214)  
at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.close(EsOutputFormat.java:196)  
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.close(ReduceTask.java:550)  
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:629)  
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)  
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)  
at java.security.AccessController.doPrivileged(Native Method)  
at javax.security.auth.Subject.doAs(Subject.java:415)  
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)  
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

The important pieces of my code are:

... implements Tool {  
Configuration conf = getConf();  
conf.setBoolean(MRJobConfig.MAP\_SPECULATIVE,false);  
conf.set("es.nodes", args[1]);  
conf.set("es.resource", args[2]);  
conf.set("es.input.json", "yes");  
conf.set(AvroJob.INPUT\_SCHEMA, Occurrence.getClassSchema().toString());  
conf.setBoolean(MRJobConfig.MAPREDUCE\_JOB\_USER\_CLASSPATH\_FIRST, true);  
conf.setBoolean(MRJobConfig.MAPREDUCE\_TASK\_CLASSPATH\_PRECEDENCE, true);

Job job = Job.getInstance(conf,"occurrence-es-indexing");  
job.setUserClassesTakesPrecedence(true);

job.setInputFormatClass(AvroKeyInputFormat.class);

job.setMapOutputKeyClass(NullWritable.class);  
job.setMapOutputValueClass(BytesWritable.class);

job.setMapperClass(OccurrenceAvroMapper.class);  
job.setOutputFormatClass(EsOutputFormat.class); ...}

...extends Mapper .. {  
@Override  
public void map(AvroKey occurrenceAvro, NullWritable value, Context context) throws IOException, InterruptedException {  
context.write(NullWritable.get(),new BytesWritable(GSON.toJson(occurrenceAvro.datum()).getBytes()));  
}  
}

---

<div class="post-metadata">

**Author:** ![thn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thn/32/8061_2.png) [@thn](https://discuss.elastic.co/u/thn)\
**Post date:** [May 11, 2016, 3:22pm UTC](https://discuss.elastic.co/t/mapreduce-json-indexing/49778/2 "2016-05-11T15:22:04Z")

</div>

> [@fede](#):
>
> Error: org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: Found unrecoverable error [130.226.238.240:9200] returned Bad Request(400) - failed to parse;mapper [values] of different type, current\_type [long], merged\_type [string]; Bailing out..

based on this error message, you might want to check our JSON data against ES mapping of the index that you are indexing.

---

<div class="post-metadata">

**Author:** ![fede](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fede/32/126283_2.png) [@fede](https://discuss.elastic.co/u/fede)\
**Post date:** [May 12, 2016, 6:51am UTC](https://discuss.elastic.co/t/mapreduce-json-indexing/49778/3 "2016-05-12T06:51:37Z")

</div>

I can actually index the same data using the java API, but it doesn't work  
with the mapreduce job.

---

<div class="post-metadata">

**Author:** ![thn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thn/32/8061_2.png) [@thn](https://discuss.elastic.co/u/thn)\
**Post date:** [May 12, 2016, 10:17am UTC](https://discuss.elastic.co/t/mapreduce-json-indexing/49778/4 "2016-05-12T10:17:47Z")

</div>

I did think this error could be related to MapReduce... but I found the link below talking about "similar" issue... so I still think it has something to do with your data and ES mapping.

So check out the article below...

> **[What are mappings in elasticsearch](http://www.efroot.org/2013/06/mappings-in-elasticsearch.html)**
>
> A blog about modern web development, frameworks, tools and some cool tricks you can use to code efficiently.

Can you try with simple JSON data string, not Avro, where you can verify the mapping easily since I don't know what's in your Avro file and its JSON representation?

I'm assuming you use the document below when working on this MapReduce job.  
[https://www.elastic.co/guide/en/elasticsearch/hadoop/current/mapreduce.html](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/mapreduce.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:52pm UTC](https://discuss.elastic.co/t/mapreduce-json-indexing/49778/5 "2017-07-05T22:52:08Z")

</div>