Debugging geoip plugin - Ruby @logger statements appear but not Java logger

[hukel]

I know the geoip plugin has some limitations when handling private IP addresses so I am not trying to solve that problem. I am trying to get debug output from the plugin to explain where it is giving up when processing my IP input.

My logstash has the plugin at debug level (checked via curl)
"logstash.filters.geoip" : "DEBUG",

I am getting output from geoip.rb
[logstash.filters.geoip ][main] IP 10.99.99.99 was not found in the database {:event=>#LogStash::Event:0x5dc19e63}

    def tag_unsuccessful_lookup(event)
    @logger.debug? && @logger.debug("IP #{event.get(@source)} was not found in the database", :event => event)`

I am not seeing any of these three debug messages from GeoIPFilter.java

        } catch (UnknownHostException e) {
      logger.debug("IP Field contained invalid IP address or hostname. exception={}, field={}, event={}", e, sourceField, event);
    } catch (AddressNotFoundException e) {
      logger.debug("IP not found! exception={}, field={}, event={}", e, sourceField, event);
    } catch (GeoIp2Exception | IOException e) {
      logger.debug("GeoIP2 Exception. exception={}, field={}, event={}", e, sourceField, event);
    }

Do I need to configure something differently to capture those "logger" outputs from within the plugin's Java code?

[Badger]

Add

loggers = geoip
logger.geoip.name = org.logstash.filters.GeoIPFilter
logger.geoip.level = DEBUG
logger.geoip.appenderRef.plain_console.ref = plain_console
logger.geoip.additivity = false

to /etc/logstash/log4j2.properties

Adjust the appenderRef as needed.

[hukel]

Many thanks. I tried searching for that answer but I was getting lost in a sea of suggestions about reading/ingesting log4j.

I used your answer to get to this log4j.properties config:

# Ruby class
logger.geoipruby.name = logstash.filters.geoip
logger.geoipruby.level = debug

# Java class
logger.geoipjava.name = org.logstash.filters.GeoIPFilter
logger.geoipjava.level = DEBUG
logger.geoipjava.appenderRef.plain_console.ref = plain_console

That gives me the extra messages I need:

[DEBUG][org.logstash.filters.GeoIPFilter][main] IP Field contained invalid IP address or hostname. exception=java.net.UnknownHostException: 10.99.6.X, field=[source][ip], event=2020-11-08T00:08:56.252Z XXXX %{message}
[DEBUG][logstash.filters.geoip   ][main] IP 10.99.6.X was not found in the database {:event=>#<LogStash::Event:0x3727f106>}

[DEBUG][org.logstash.filters.GeoIPFilter][main] GeoIP2 Exception. exception=com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.util.ArrayList` out of START_OBJECT token
 at [Source: UNKNOWN; line: -1, column: -1] (through reference chain: com.maxmind.geoip2.model.CityResponse["subdivisions"]), field=[source][ip], event=2020-11-08T00:08:56.123Z XXXX %{message}
[DEBUG][logstash.filters.geoip   ][main] IP 10.99.6.20 was not found in the database {:event=>#<LogStash::Event:0x6e6ef3a3>}