GeoIP2 plugin install throws an error on LS 5.0

Elastic Stack Logstash
1

After upgrading to Logstash 5 I am unable to install the geoip2 plugin (to handle IPv6 addresses). Any help would be great.

Error output:

    ./logstash-plugin install logstash-filter-geoip2 
Validating logstash-filter-geoip2
Installing logstash-filter-geoip2
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core-plugin-api":
  In snapshot (Gemfile.lock):
    logstash-core-plugin-api (= 2.1.16)

  In Gemfile:
    logstash-devutils (~> 1.1) java depends on
      logstash-core-plugin-api (~> 2.0) java

    .... <truncated because of post limit, listed all installed plugins/versions>

    logstash-filter-geoip2 (>= 0) java depends on
      logstash-core-plugin-api (~> 1.0) java

    logstash-core-plugin-api (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
2

After some research it seems the built-in geoip filter should support IPv6 (as noted here: https://github.com/logstash-plugins/logstash-filter-geoip/issues/33), however I am just getting a flood of log entries stating

reason"=>"failed to parse [src_ip]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse ip [valid IPv6 address removed], not a valid ipv4 address (4 dots)"}}}}}

If the newer geoip plugin supports IPv6 is there something I need to change in my config?

Here is the current geoip block in my filter config, and it seems to work fine on IPv4 addresses.

      geoip {
        add_tag => [ "GeoIP" ]
        database => "/etc/logstash/GeoLite2-City.mmdb" 
        source => "src_ip"
        }
3

That error message is from Elasticsearch, not Logstash. It looks like src_ip has been mapped as an IP address but doesn't contain an IP address. What's in the field?

4

Well, I'm not exactly sure what has happened, but I haven't seen this error in the logs over the last few days now. After a quick spot check it looks like logs with IPv6 address are getting geoip data added, even though it is just dead center on US without city/state, but that is Maxmind issue not Logstash.

However... Now I seem to be getting a java exception error if an IP field is blank.

2016-10-31T10:41:21,060][ERROR][logstash.filters.geoip ] IP Field contained invalid IP address or hostname {:exception=>java.net.UnknownHostException: -: Name or service not known,

Any idea how i can "ignore" the null values before they are parsed by geoip? I figure some sort of "IF" statement, but I'm not sure how to filter on field = Null.

I will keep an eye on the logs to see if the parse issue comes back, but for now all seems ok.

5

It appears the field isn't null but contains a hyphen. In that case a conditional looks like this:

if [name-of-ip-address-field] != "-" {
  geoip { ... }
}

You could also do a regexp match only use the geoip filter if the field in question looks like an IP address.