A while ago, I made a Logstash filter that extracts geolocation information from the client IP in Nginx logs, like this:
geoip {
source => "clientip"
target => "geoip"
database => "/etc/logstash/GeoLiteCity.dat"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
Then I had to modify the Filebeat index template that was loaded into ElasticSearch, in order to mark the geoip.location field with the type geo_point. The index template I've used is this:
{
"mappings": {
"_default_": {
"_all": {
"enabled": true,
"norms": {
"enabled": false
}
},
"dynamic_templates": [
{
"template1": {
"mapping": {
"doc_values": true,
"ignore_above": 1024,
"index": "not_analyzed",
"type": "{dynamic_type}"
},
"match": "*"
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"message": {
"type": "string",
"index": "analyzed"
},
"offset": {
"type": "long",
"doc_values": "true"
},
"geoip" : {
"type" : "object",
"dynamic": true,
"properties" : {
"location" : { "type" : "geo_point" }
}
}
}
}
},
"settings": {
"index.refresh_interval": "5s"
},
"template": "filebeat-*"
}
More recently I've used the json plugin in logstash to extract a JSON document into a field, from a different log source, like this:
json {
source => "data"
target => "node_post_data_json"
}
This has created the field node_post_data_json.geocode.coordinates, which I would like to mark as type geo_point. But the problem is, the geo_point field is three levels down, instead of two, and I'm not sure what is the correct syntax in the template for that. I've tried the template shown below, but it doesn't seem to do anything. What is the correct syntax?
{
"mappings": {
"_default_": {
"_all": {
"enabled": true,
"norms": {
"enabled": false
}
},
"dynamic_templates": [
{
"template1": {
"mapping": {
"doc_values": true,
"ignore_above": 1024,
"index": "not_analyzed",
"type": "{dynamic_type}"
},
"match": "*"
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"message": {
"type": "string",
"index": "analyzed"
},
"offset": {
"type": "long",
"doc_values": "true"
},
"geoip" : {
"type" : "object",
"dynamic": true,
"properties" : {
"location" : { "type" : "geo_point" }
}
},
"node_post_data_json": {
"type" : "object",
"dynamic": true,
"properties" : {
"geocode": {
"type" : "object",
"dynamic": true,
"properties" : {
"coordinates" : { "type" : "geo_point" }
}
}
}
}
}
}
},
"settings": {
"index.refresh_interval": "5s"
},
"template": "filebeat-*"
}