Index twitter data with coordinates geo_point parse exception

Elastic Stack Elasticsearch
1

Hello,

I'm trying to index some twitter data without success because of the coordinates field that I want to use as geo_point.

Here is my template:

Here is a document:

When I run steps to delete index, replace template and insert my document I'm getting an error:

{"error":{"root_cause":[{"type":"parse_exception","reason":"geo_point expected"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"parse_exception","reason":"geo_point expected"}},"status":400}

Steps:

HOSTNAME=localhost
curl -XDELETE http://$HOSTNAME:9200/ul-twitter-test
curl -XPUT http://$HOSTNAME:9200/_template/ul-twitter     -d @templates/ul-twitter.basic.json
curl -XPUT http://$HOSTNAME:9200/ul-twitter-test/tweets/1 -d @samples/tweet-with-geopoint.json

Currently running with version 5.6.4.

Any clues?

Thanks
Bruno

2

Hey,

you are actually not indexing a geo_point (that consists only of a lon and lat pair), but a geo_shape, which in your example incidentally turns out to be a point, but could also be a line or a polygon.

In order to fix this, you could just use this in your tweet

coordinates: { lat: 11, lon: 12 }

and then everything should work

--Alex

3

Hello @spinscale,

Thanks for your response, but as per documentation geo_point type can be passed as an array:
https://www.elastic.co/guide/en/elasticsearch/reference/5.6/geo-point.html

Snippet:

PUT my_index/my_type/4
{
  "text": "Geo-point as an array",
  "location": [ -71.34, 41.12 ] 
}

What am I missing?

4

nothing. Just because it works, does not mean it is intuitive to me :slight_smile:

The person coming after you has to know which is the latitude and the longitude. And this person also needs to know, if you write a geo point as a string like41.12, -71.34, you need to reverse it to make it work. Because of that I am just always preferring to mark, which field represents what by naming it appropriately. Internally everyone of those three representations gets stored the same for, so it does not make a difference when searching.

personal preference, you may ignore :slight_smile:

5

Yes, makes sense about personal preferences, but I meant that the tweet payload comes with this:

  "coordinates" : {
    "type" : "Point",
    "coordinates" : [ -84.51194444, 37.47361111 ]
  },

and my mapping template for coordinates.coordinates type is geo_point and it doesn't work... as this representation is normally correctly handled by elasticsearch (as per doc).

.... I'm really puzzled

Thanks

6

no, this representation only works for a geo shape, a point needs to be modelled like this

coordinates: [ -84.51194444, 37.47361111 ]

coordinates: "37.47361111, -84.51194444, "

or

coordinates: { lon: 37.47361111, lat: -84.51194444, }

the structure that you got resembles a geo shape because it contains the type and coordinates field inside of the coordinates field.

hope this makes sense

7

Sorry, not clear yet... :thinking:

The field coordinates.coordinates is an array, and by the template this field is defined to be a geo_point, and per doc an array is a valid representation of geo_point...

is it because it must not be in an object? on the root?

8

I had same problem liked yours.
And I fixed the problem by the script below.
Hope that works for you too.

9

Thanks @Jim_Weng you are right!

Still don't get how is Elasticsearch is doing things between your template and mine: very simple details! All this in a curly brace party. :slight_smile:

You saved me hours.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.