Getting geo_point from Twitter

Elastic Stack Logstash
1

Afternoon all, I've been using logstash to feed twitter data into elasticsearch so I can visualise it with kibana. Have successfully changed config file to work with last week's new releases, so no problems there, but I'm having a tricky time working with the mappings.

Whenever I read the data into elasticsearch, coordinates.coordinates is always a double in kibana, making it impossible to visualise on a tile map. I've created endless different mapping files, confirmed that they are being loaded ok, and brought in tweets with the necessary data in that field. But Kibana refuses to recognise it as anything other than a double. See below for my current conf and json mapping text.

Have read topics such as Mapping for geo_point-usage not working with twitter input and every other one I could find, but none of their solutions seem to work.

input { twitter { consumer_key => "removed" consumer_secret => "removed" oauth_token => "removed" oauth_token_secret => "removed" keywords => ["London"] full_tweet => true } } output { elasticsearch { hosts => ["localhost"] index => "london" document_type => "tweet_ls" template => "location of template.json" template_name => "twitter" } }

{ "template" : "twitter", "settings" : { "index.refresh_interval" : "5s" }, "mappings" : { "_default_" : { "_all" : {"enabled" : true, "omit_norms" : true}, "dynamic_templates" : [ { "message_field" : { "match" : "message", "match_mapping_type" : "string", "mapping" : { "type" : "string", "index" : "analyzed", "omit_norms" : true } } }, { "string_fields" : { "match" : "*", "match_mapping_type" : "string", "mapping" : { "type" : "string", "index" : "analyzed", "omit_norms" : true, "fields" : { "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256} } } } } ], "properties" : { "@version": { "type": "string", "index": "not_analyzed" }, "coordinates" : { "type" : "object", "dynamic" : true, "properties" : { "coordinates" : { "type" : "geo_point" } } }, "geoip" : { "type" : "object", "dynamic": true, "properties" : { "location" : { "type" : "geo_point" } } } } } } }

Any guidance would be appreciated, been struggling with this for a couple of days now and getting nowhere. Others have definitely got the coordinates.coordinates field working as geo_points in this way but mine just won't.

Thanks,

James

2

Your index template applies to indexes matching the pattern "twitter" but the actual name of your index is "london" so your index template won't apply.

3

Can't believe it was that simple, thank you so much!

4

Hi,

I am new to ELK and hence now aware of most of the concepts. Still learning.

I went through your discussion.

I am also fetching data from twitter and followed how you had done it.
But now where is this location or coordinates getting updated? I am not able to figure out.

My use case is that i want to show on a map - some heat map kind of thing where all tweets have come up.

Please can you suggest something.

Thanks