Afternoon all, I've been using logstash to feed twitter data into elasticsearch so I can visualise it with kibana. Have successfully changed config file to work with last week's new releases, so no problems there, but I'm having a tricky time working with the mappings.
Whenever I read the data into elasticsearch, coordinates.coordinates is always a double in kibana, making it impossible to visualise on a tile map. I've created endless different mapping files, confirmed that they are being loaded ok, and brought in tweets with the necessary data in that field. But Kibana refuses to recognise it as anything other than a double. See below for my current conf and json mapping text.
Have read topics such as Mapping for geo_point-usage not working with twitter input and every other one I could find, but none of their solutions seem to work.
input {
twitter {
consumer_key => "removed"
consumer_secret => "removed"
oauth_token => "removed"
oauth_token_secret => "removed"
keywords => ["London"]
full_tweet => true
}
}
output {
elasticsearch {
hosts => ["localhost"]
index => "london"
document_type => "tweet_ls"
template => "location of template.json"
template_name => "twitter"
}
}
{
"template" : "twitter",
"settings" : {
"index.refresh_interval" : "5s"
},
"mappings" : {
"_default_" : {
"_all" : {"enabled" : true, "omit_norms" : true},
"dynamic_templates" : [ {
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true
}
}
}, {
"string_fields" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true,
"fields" : {
"raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
}
}
}
} ],
"properties" : {
"@version": { "type": "string", "index": "not_analyzed" },
"coordinates" : {
"type" : "object",
"dynamic" : true,
"properties" : {
"coordinates" : { "type" : "geo_point" }
}
},
"geoip" : {
"type" : "object",
"dynamic": true,
"properties" : {
"location" : { "type" : "geo_point" }
}
}
}
}
}
}
Any guidance would be appreciated, been struggling with this for a couple of days now and getting nowhere. Others have definitely got the coordinates.coordinates field working as geo_points in this way but mine just won't.
Thanks,
James