# Problems with Geo\_point type and doc\_values **URL:** \ **Category:** Elasticsearch\ **Created:** [August 28, 2015, 3:50pm UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256 "2015-08-28T15:50:14Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![clement\_tourriere](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/clement_tourriere/32/479_2.png) [@clement\_tourriere](https://discuss.elastic.co/u/clement_tourriere)\ **Post date:** [August 28, 2015, 3:50pm UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/1 "2015-08-28T15:50:15Z")
Hi all, I have a very strange problem with geo\_point type and "script\_fields" or "fielddata\_fields" search queries. What I want to do is to retrieve latitude and longitude from a geo\_point field created by a custom plugin that computes and indexes the centroid of a shape ([https://github.com/opendatasoft/elasticsearch-plugin-geoshape](https://github.com/opendatasoft/elasticsearch-plugin-geoshape)). By definition, I don't have this geo\_point in the document source, in order to get it in the result, I need to use fielddata\_fields or script\_fields. 1 ) fielddata\_fields It's working fine with one node (or one shard), but, since geo\_point type can't be serialized an exception is raised with two nodes or more. It is actually a bug or not ? A working patch could be : [https://gist.github.com/clement-tourriere/2aa7219bd1da96393cbd](https://gist.github.com/clement-tourriere/2aa7219bd1da96393cbd). 2 ) Using script\_fields When using doc['my\_shape.centroid'].value, I have the same problem as above (same code used internally with getScriptValue()). When using doc['my\_shape.centroid'].lat and doc['my\_shape.centroid'].lon, it's working just fine with simple request. When performing parallel queries on 10000 thousands documents minimum, it's raising an exception : [https://gist.github.com/clement-tourriere/855a24f150efb06e8fec](https://gist.github.com/clement-tourriere/855a24f150efb06e8fec). I have made a lot of tests to be able to reproduced it, and here's my conclusion. I have started with indexing a geo\_point field with doc\_values set to true. A full example can be found here (writing in python with celery for parallel indexing/requests) : [https://gist.github.com/clement-tourriere/82b27c25d5876ae53eef](https://gist.github.com/clement-tourriere/82b27c25d5876ae53eef) Everything is working just fine. The next example is using a geo\_point defined as an external\_value and doc\_value set to true. (I have write a simple class for that, or you can use the geo\_shape plugin to test this case) : [https://gist.github.com/clement-tourriere/1ffe9eba5bfa0070e5d2](https://gist.github.com/clement-tourriere/1ffe9eba5bfa0070e5d2). With the same use case (10 parallel requests on 100000 documents), it is now raising the groovy bufferOverflow exception. If I remove doc\_value from external geo\_point definition, it's working fine again. Could you please help me with that. Thank you again for your incredible work. Regards, Clément ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 29, 2015, 10:35pm UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/2 "2015-08-29T22:35:51Z")
Regarding #1, can you provide the error you see? ---
**Author:** ![clement\_tourriere](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/clement_tourriere/32/479_2.png) [@clement\_tourriere](https://discuss.elastic.co/u/clement_tourriere)\ **Post date:** [August 31, 2015, 8:15am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/3 "2015-08-31T08:15:24Z")
Sorry, I didn't put a gist to reproduce it (for the #1). The error is : ``` "RemoteTransportException[[Yukon Jack][inet[/192.168.1.43:9301]][indices:data/read/search[phase/fetch/id]]]; nested: IOException[Can't write type [class org.elasticsearch.common.geo.GeoPoint]]; " ``` Here the gist : [https://gist.github.com/clement-tourriere/8d06a0985f859666ae53](https://gist.github.com/clement-tourriere/8d06a0985f859666ae53) ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 31, 2015, 8:27am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/4 "2015-08-31T08:27:13Z")
I just did that with 3 points, but no exception. What version are you on? ---
**Author:** ![clement\_tourriere](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/clement_tourriere/32/479_2.png) [@clement\_tourriere](https://discuss.elastic.co/u/clement_tourriere)\ **Post date:** [August 31, 2015, 8:43am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/5 "2015-08-31T08:43:23Z")
I have tried with elasticsearch 1.6.2 and 2.0 beta1. You need to have a cluster with a least two nodes for seeing the exception. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 31, 2015, 9:27am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/6 "2015-08-31T09:27:11Z")
Do you still get it with ES 1.7.1? ---
**Author:** ![clement\_tourriere](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/clement_tourriere/32/479_2.png) [@clement\_tourriere](https://discuss.elastic.co/u/clement_tourriere)\ **Post date:** [August 31, 2015, 9:42am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/7 "2015-08-31T09:42:51Z")
Yes I still get it with es 1.7.1 The full stack is : ``` org.elasticsearch.transport.RemoteTransportException: [Scream][inet[/192.168.1.43:9301]][indices:data/read/search[phase/fetch/id]] Caused by: java.io.IOException: Can't write type [class org.elasticsearch.common.geo.GeoPoint] at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:415) at org.elasticsearch.search.internal.InternalSearchHitField.writeTo(InternalSearchHitField.java:110) at org.elasticsearch.search.internal.InternalSearchHit.writeTo(InternalSearchHit.java:726) at org.elasticsearch.search.internal.InternalSearchHits.writeTo(InternalSearchHits.java:259) at org.elasticsearch.search.fetch.FetchSearchResult.writeTo(FetchSearchResult.java:104) at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:99) at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:76) at org.elasticsearch.search.action.SearchServiceTransportAction$FetchByIdTransportHandler.messageReceived(SearchServiceTransportAction.java:869) at org.elasticsearch.search.action.SearchServiceTransportAction$FetchByIdTransportHandler.messageReceived(SearchServiceTransportAction.java:862) at org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.doRun(MessageChannelHandler.java:279) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) ``` ---
**Author:** ![clement\_tourriere](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/clement_tourriere/32/479_2.png) [@clement\_tourriere](https://discuss.elastic.co/u/clement_tourriere)\ **Post date:** [September 1, 2015, 9:18am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/8 "2015-09-01T09:18:38Z")
I can reproduce the #1 problem all the time, that is really strange it doesn't happen for you. From scratch : - download elasticsearch and untar it - run two instances of elasticsearch (on the same machine) - run the gist : [https://gist.github.com/clement-tourriere/8d06a0985f859666ae53](https://gist.github.com/clement-tourriere/8d06a0985f859666ae53) - You maybe need to run the search request twice to see the error And for the #2 error, any hint ? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 1, 2015, 11:21am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/9 "2015-09-01T11:21:17Z")
I haven't had time to test this with 2 nodes sorry, I'll try tomorrow. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 7, 2015, 6:40am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/10 "2015-09-07T06:40:17Z")
I replicated this on 1.7.1. I'll ask one of the geo team to take a gander and comment 🙂 ---
**Author:** ![Ben\_George](https://avatars.discourse-cdn.com/v4/letter/b/58956e/32.png) [@Ben\_George](https://discuss.elastic.co/u/Ben_George)\ **Post date:** [September 7, 2015, 8:21am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/11 "2015-09-07T08:21:32Z")
A work around for those needing one: Add lat\_lon: true for your geo\_point field mapping and retrieve them using field\_name.lat / field\_name.lon as normal fields. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 7, 2015, 9:29am UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/12 "2015-09-07T09:29:15Z")
I can see you've already raised this - [https://github.com/elastic/elasticsearch/issues/13340](https://github.com/elastic/elasticsearch/issues/13340) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:51pm UTC](https://discuss.elastic.co/t/problems-with-geo-point-type-and-doc-values/28256/13 "2017-07-05T23:51:49Z")