Logstash Twitter Input with Coordinates is not working

Elastic Stack Logstash
1

Hello together,

I am using Logstash in order to ingest data from Twitter.
I am using the following configuration file:
input {
twitter {
consumer_key => "XXXXXXXXXX"
consumer_secret => "XXXXXXXXXX"
oauth_token => "XXXXXXXXXX"
oauth_token_secret => "XXXXXXXXXX"
keywords => [ "Big Data", "Elasticsearch", "Logstash", "Kibana", "Elastic", "ELK stack" ]
full_tweet => true
}
}
filter {}
output {
stdout { codec => dots }
elasticsearch {
hosts => "localhost:9200"
index => "twitter"
document_type => "tweet"
template => "/usr/local/Cellar/logstash/twitter_template.json"
template_name => "twitter"
}
}
And the following template:
{
"template" : "twitter",
"settings" : {
"index.refresh_interval" : "1s"
},
"mappings" : {
"default" : {
"_all" : {"enabled" : true, "omit_norms" : true},
"dynamic_templates" : [ {
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true
}
}
}, {
"string_fields" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true,
"fields" : {
"raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
}
}
}
} ],
"properties" : {
"@version": { "type": "string", "index": "not_analyzed" },
"coordinates" : {
"type" : "object",
"dynamic" : true,
"properties" : {
"coordinates" : { "type" : "geo_point" }
}
},
"geoip" : {
"type" : "object",
"dynamic": true,
"properties" : {
"location" : { "type" : "geo_point" }
}
}
}
}
}
}

However, Kibana does not provide any information about the coordinates. It is clearly that not all of the users will use it, but normally some of them should provide coordinates. Kibana says for this field: "This field is present in your elasticsearch mapping but not in any documents in the search results. You may still be able to visualize or search on it." Do you have any suggestions where the problem may occur? Furthermore, I cannot see the coordinates.coordinates field which is shown in the settings page of Kibana. The coordinates field has a questionmark is icon and not the geolocation icon.

Thank you very much in advance.

Best regards

2

Try making a tweet that matches one of those keywords and with geolocation set to see if it works.

3

Hello Mark,

I already tried this out. I think it must be something internal, since this code worked like for 1 month ago.

4

Same for me :
Config kibana Version: 5.3.0
Any change to get it work

kibana-5.4.0-twitter-error-geohash.png803×693 67.8 KB

Coordinate are indexed as a Number :

"bounding_box": {
    "coordinates": [
      [
        [
          2.224101,
          48.815521
        ],
        [
          2.224101,
          48.902146
        ],
        [
          2.469905,
          48.902146
        ],
        [
          2.469905,
          48.815521
        ]
      ]
    ],