After a long time away from this issue, I've managed to get the right setup.
In my logstash config, after I've called geoip {} I look for my IP address ranges and manually repopulate the various fields. I'm effectively declaring a new country.
if [srcip] =~ /^139\.80\./ or [srcip] =~ /^10\./ {
mutate { replace => { "[geoip][timezone]" => "Pacific/Auckland" } }
mutate { replace => { "[geoip][country_name]" => "University of Otago" } }
mutate { replace => { "[geoip][country_code2]" => "UO" } }
mutate { replace => { "[geoip][country_code3]" => "UoO" } }
mutate { remove_field => [ "[geoip][location]" ] }
mutate { add_field => { "[geoip][location]" => "170.525" } }
mutate { add_field => { "[geoip][location]" => "-45.865" } }
mutate { convert => [ "[geoip][location]", "float" ] }
mutate { replace => [ "[geoip][latitude]", -45.856 ] }
mutate { convert => [ "[geoip][latitude]", "float" ] }
mutate { replace => [ "[geoip][longitude]", 170.525 ] }
mutate { convert => [ "[geoip][longitude]", "float" ] }
}
To help keep me honest (actually, to make sure that all my settings are visible in the configuration) I explicitly set a template when writing to ES, rather than rely on state that's already there.
elasticsearch { embedded => "false"
cluster => "myclustername"
protocol => "transport"
host => "indexing host"
index => "test" # index name must be in lowercase!
template => "/etc/logstash/template.d/test"
template_name => "test"
template_overwrite => "true"
}
(Every time I drop this index, the new template will be used to re-create it.)
For my purposes, I'm using the template to switch off string analyzing (most of the fields are log data and analysis doesn't help), and to make sure I get geo_points done properly. Here's the full template from the dev box :-
{
"order" : 0,
"template" : "test*",
"settings" : { "index.refresh_interval" : "5s" },
"mappings" : {
"_default_" : {
"dynamic_templates" : [
{
"message_field" : {
"mapping" : { "index" : "analyzed", "omit_norms" : true, "type" : "string" },
"match_mapping_type" : "string",
"match" : "message" }
},
{
"string_fields" : {
"mapping" : { "index" : "not_analyzed", "ignore_above" : 256, "type" : "string" },
"match_mapping_type" : "string",
"match" : "*" }
}
],
"properties" : {
"geoip" : {
"dynamic" : true,
"path" : "full",
"properties" : { "location" : { "type" : "geo_point" } },
"type" : "object" },
"@version" : { "index" : "not_analyzed", "type" : "string" }
},
"_all" : { "enabled" : true }
}
},
"aliases" : { }
}
The end result is a custom country, with a geo-location that separates my network from the users in the same city (although the point on the map when you zoom in isn't as accurate as I want it to be, I don't think I'll worry about that!)
