# Writing rich datatypes from Pig to ElasticSearch type with mapping (Geoshape)

**URL:** <https://discuss.elastic.co/t/writing-rich-datatypes-from-pig-to-elasticsearch-type-with-mapping-geoshape/51016>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [May 26, 2016, 5:13am UTC](https://discuss.elastic.co/t/writing-rich-datatypes-from-pig-to-elasticsearch-type-with-mapping-geoshape/51016 "2016-05-26T05:13:30Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![liadl](https://avatars.discourse-cdn.com/v4/letter/l/f6c823/32.png) [@liadl](https://discuss.elastic.co/u/liadl)\
**Post date:** [May 26, 2016, 5:13am UTC](https://discuss.elastic.co/t/writing-rich-datatypes-from-pig-to-elasticsearch-type-with-mapping-geoshape/51016/1 "2016-05-26T05:13:30Z")

</div>

Hello,  
I am trying to write a Geoshape from Pig to ElasticSearch using EsStorage.  
(The Index has a mapping for the geoshape field)

I am able to write a document with that field identified as a "GeoShape" using "curl", however from Pig it fails on:  
**Caused by: org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: Found unrecoverable error [...] returned Bad Request(400) - failed to parse [affectedGeoArea];shape must be an object consisting of type and coordinates; Bailing out..**

Also a Pig chararray with a JSON contents didn't do the trick (as in GeoPoint), so it seems that I need to create a Tuple and describe: **type** , **coordinates** and **radius**  
I read the excellent documentation page at [https://www.elastic.co/guide/en/elasticsearch/hadoop/current/pig.html](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/pig.html), however I could not find a code example.

1. Sample input record from the input file looks like that:  
**circle|[-45.0,45.0]**

2. Code sample: (which fails with the message mentioned here)

_REGISTER ./elasticsearch-hadoop-2.2.0.jar;_

_loadedRecords = LOAD 'inputFile.csv' USING PigStorage('|') AS (type:chararray,coordinates:chararray);_

_elasticData = foreach loadedRecords GENERATE (type,coordinates) AS affectedGeoArea:tuple(type:chararray,coordinates:chararray);_

_DESCRIBE elasticData ;_

_DUMP elasticData;_

_STORE elasticData INTO 'myindex/mytype' USING org.elasticsearch.hadoop.pig.EsStorage('es.http.retries=10','es.nodes=localhost','es.index.auto.create=true','es.mapping.pig.tuple.use.field.names=false');_

Appreciate if you can share a Pig code example or even better add it to the documentation.

Thanks

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [June 1, 2016, 12:17pm UTC](https://discuss.elastic.co/t/writing-rich-datatypes-from-pig-to-elasticsearch-type-with-mapping-geoshape/51016/2 "2016-06-01T12:17:32Z")

</div>

1. consider using the latest stable version of ES-Hadoop, namely 2.3.x
2. enable logging (also in the docs) to see the REST HTTP queries generated. This is a great way to map things back to ES.
3. when using tuples (as you are), disable the use of field names (which you are)

Hope this helps,

---

<div class="post-metadata">

**Author:** ![liadl](https://avatars.discourse-cdn.com/v4/letter/l/f6c823/32.png) [@liadl](https://discuss.elastic.co/u/liadl)\
**Post date:** [June 1, 2016, 1:43pm UTC](https://discuss.elastic.co/t/writing-rich-datatypes-from-pig-to-elasticsearch-type-with-mapping-geoshape/51016/3 "2016-06-01T13:43:16Z")

</div>

Thanks Costin,  
We debugged this both on the es-hadoop (client) and ES server side, still we didn't find a suitable datatype in Pig to represent a GeoShape (I can explain some more if someone is interested)

Anyway we changed our Pig script to build a **single JSON string** from all the relevant input fields (using a UDF), then we added the EsStorage property: **'es.input.json=true'**

STORE elasticData INTO 'myindex/mytype' USING org.elasticsearch.hadoop.pig.EsStorage('es.http.retries=10','es.nodes=localhost','es.index.auto.create=true','es.mapping.pig.tuple.use.field.names=false','es.input.json=true');

That did the trick and the document was properly saved containing a GeoShape field 🙂

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [June 1, 2016, 2:06pm UTC](https://discuss.elastic.co/t/writing-rich-datatypes-from-pig-to-elasticsearch-type-with-mapping-geoshape/51016/4 "2016-06-01T14:06:50Z")

</div>

By the way, have you seen this thread - looks to be directly related to the same issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:24pm UTC](https://discuss.elastic.co/t/writing-rich-datatypes-from-pig-to-elasticsearch-type-with-mapping-geoshape/51016/5 "2017-07-06T13:24:31Z")

</div>