So I was giving the 2.3 release a spin and am running into an issue with the geoip filter that was working fine in 2.2.1.
The latitude for every single processed event is the same invalid value. If i stop and restart logstash, the value changes but is the same for all failed events.
I have a copy of 2.2.1 running with the same config and maxmind database and it's functioning fine.
if [targetip] != "-" {
geoip {
source => "targetip"
target => "geoip"
database => "C:\logstash\maxmind\GeoLiteCity.dat"
add_field => ["coords", "%{[geoip][longitude]}"]
add_field => ["coords", "%{[geoip][latitude]}"]
}
}
Elasticsearch refuses to index the event, with the following error:
response=>{"create"=>{"_index"=>"sophosproxy-2016.03.30", "_type"=>"syslog", "_id"=>"AVPI53FjYgO7CyFydcBk", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"illegal latitude value [269.95656967163086] for coords"}}}}, :level=>:warn}
I didn't note any changes in the geoip filter in this new release, so I'm not sure where or why it's breaking down. Any thoughts?
Running in debug mode doesn't seem to reveal much helpful (to me) info..
{:timestamp=>"2016-03-30T12:48:29.753000-0700", :message=>"filters/LogStash::Filters::GeoIP: adding value to field", :field=>"coords", :value=>["%{[geoip][longitude]}", "%{[geoip][latitude]}"], :level=>:debug, :file=>"/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.0-java/lib/logstash/util/decorators.rb", :line=>"31", :method=>"add_fields"}
{:timestamp=>"2016-03-30T12:48:29.753000-0700", :message=>"filters/LogStash::Filters::GeoIP: adding value to field", :field=>"coords", :value=>["%{[geoip][longitude]}", "%{[geoip][latitude]}"], :level=>:debug, :file=>"/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.0-java/lib/logstash/util/decorators.rb", :line=>"31", :method=>"add_fields"}
Yarg.
So I added some additional logging to the plugin to try and track things down.. it seems like the plugin is getting and returning valid values. I now have no idea where the invalid value is coming from. I don't think it's the plugin anymore.
log containing result object just before it's returned:
{:level=>:error, "ip"=>"216.239.38.120", "country_code2"=>"US", "country_code3"=>"USA", "country_name"=>"United States", "continent_code"=>"NA", "region_name"=>"CA", "city_name"=>"Mountain View", "postal_code"=>"94043", "latitude"=>37.41919999999999, "longitude"=>-122.0574, "dma_code"=>807, "area_code"=>650, "timezone"=>"America/Los_Angeles", "real_region_name"=>"California", "location"=>[-122.0574, 37.41919999999999]}`