Failure running tests for logstash-filter-geoip plugin - logging configuration not loading

Elastic Stack Logstash
1

I cloned the geoip plugin source from github:

https://github.com/logstash-plugins/logstash-filter-geoip/commit/921d1cd

Followed the README to install the pre-reqs:

bundle install
bundle exec rake vendor

Then ran the tests:

bundle exec rspec

But I got the following error:

--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.4 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.0 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.module:jackson-module-afterburner already loaded with version 2.7.4 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.4 - omit version 2.9.1
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

An error occurred while loading ./spec/filters/geoip_spec.rb.
Failure/Error: # encoding: utf-8

ArgumentError:
  invalid level[OFF] for logger[]
# ./spec/filters/geoip_spec.rb:1:in `<main>'
# ./spec/filters/geoip_spec.rb:2:in `(root)'
# ------------------
# --- Caused by: ---
# NoMethodError:
#   undefined method `getConfiguration' for nil:NilClass
#   ./spec/filters/geoip_spec.rb:1:in `<main>'
No examples found.


Finished in 0.00555 seconds (files took 2.74 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

A debugged a bit and what's happening is that the code is reaching line 97 of logger.rb:

jruby-9.1.13.0/lib/ruby/gems/shared/gems/logstash-core-5.5.1.snapshot1-java/lib/logstash/logging/logger.rb:97

Which looks like this:

# Clone of org.apache.logging.log4j.core.config.Configurator.setLevel(), but using initialized @@logging_context
      def self.set_level(_level, path)
        configuration = @@logging_context.getConfiguration()
        level = Level.valueOf(_level)
        if path.nil? || path.strip.empty?

And @@logging_context is nil at this point.

It's like self.initialize(config_location) in that file is never called for some reason?

def self.initialize(config_location)
        @@config_mutex.synchronize do
          if @@logging_context.nil?

Not sure if this is relevant, but here's the output of bundle exec rake vendor:

Downloading https://s3.amazonaws.com/download.elasticsearch.org/logstash/maxmind/GeoLite2-ASN.mmdb
Downloading https://s3.amazonaws.com/download.elasticsearch.org/logstash/maxmind/LICENSE.txt
-------------------> Wrote /Users/jesse/Devel/ubercraft/source/logstash-filter-geoip/gradle.properties
logstashCoreGemPath=/Users/jesse/bin/jruby-9.1.13.0/lib/ruby/gems/shared/gems/logstash-core-5.5.1.snapshot1-java
logstashCoreEventGemPath=Could not find gem 'logstash-core-event'.
Starting a Gradle Daemon (subsequent builds will be faster)
:generateGemJarRequiresFile
:compileJava
:processResources UP-TO-DATE
:classes
:jar
:vendor

Can anyone tell me what I'm doing wrong?

Thanks,
Jesse.

2

Hi,

Which version of Logstash you are using? and also can you give more clarification on your ques as geoip plugin is inbuilt with logstash.

3

Hi,

All I'm trying to do is checkout the code for the logstash geoip filter, build it locally, then run the tests. As I mentioned this is the commit I'm trying to build/test:

The reason is I'd like to understand the code and then hopefully use it as a good example for how to implement my own filters.

But I can't even get that far. Any help with this appreciated.

Thanks,
Jesse.

4

Hi,

I have the same problem doing the test.

Here is the pipeline of a working commit, it passed last week but when I ran it again today, it gave the same error as what you have. I supposed this has something to do with bundle install.

5

Thanks. So this issue is caused by something external to the repo?

Here's a copy of my shell, going through the sequence of cloning, setting up and running the tests (as documented in the README.md file):

Does anyone know how to get this working?

Cheers,
Jesse.

6

I've just tried the same thing with a different plugin:

Checked out the source, installed deps, build and test, and it has the same problem.

So is this some problem with my dev environment? I'm just doing this on my mac, just local, not using Docker or any VM. With JRuby version 9.1.13.0.

7

I got around this problem by dowgrading to logstash-devutils version 1.3.4. Apparently the latest version 1.3.5 has some bugs that cause this.

8

Thanks kompa3. Just downgraded and locked-in to 1.3.4. Will want to open up to newer (working) versions when available.

9

I'm experiencing a similar problem with travis check jobs for a small patch to logstash-filter-elasticsearch: https://github.com/logstash-plugins/logstash-filter-elasticsearch/pull/81

But I simply replaced a word in an asciidoc file...

10

I solved the issue by using local logstash-core and specify the path to it in Gemfile.

https://git.bitsensor.io/plugins/logstash/blob/master/Gemfile

11

Thanks @jweite, I tried this suggestion and the tests now run!

jesse ~/source/logstash-filter-geoip (master)
> git diff
diff --git a/logstash-filter-geoip.gemspec b/logstash-filter-geoip.gemspec
index de8a1ad..b994928 100644
--- a/logstash-filter-geoip.gemspec
+++ b/logstash-filter-geoip.gemspec
@@ -22,6 +22,6 @@ Gem::Specification.new do |s|

   # Gem dependencies
   s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
-  s.add_development_dependency 'logstash-devutils'
+  s.add_development_dependency 'logstash-devutils', "= 1.3.4"
   s.add_development_dependency 'benchmark-ips'
 end

Does anyone know why the all the test don't pass now? This is the commit I'm running:

63cf67a by Jake Landis - Travis - add 6.0 build, remove default JRuby 1.7 build, bump RVM versions

And this is the test output:

jesse ~/source/logstash-filter-geoip (master)
> bundle exec rspec
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.4 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.0 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.module:jackson-module-afterburner already loaded with version 2.7.4 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.4 - omit version 2.9.1
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Sending Logstash's logs to  which is now configured via log4j2.properties
Run options: exclude {:redis=>true, :socket=>true, :performance=>true, :couchdb=>true, :elasticsearch=>true, :elasticsearch_secure=>true, :export_cypher=>true, :integration=>true, :windows=>true}

Randomized with seed 14034
...............F......

Failures:

  1) LogStash::Filters::GeoIP source is derived from target when source field 'ip' is a subfield of 'target' should set other subfields of 'target' properly
     Failure/Error: expect(event.get("[target][city_name]")).to eq("Worcester")

       expected: "Worcester"
            got: "Salem"

       (compared using ==)
     # ./spec/filters/geoip_spec.rb:96:in `block in (root)'

Finished in 0.94125 seconds (files took 2.86 seconds to load)
22 examples, 1 failure

Failed examples:

rspec ./spec/filters/geoip_spec.rb:94 # LogStash::Filters::GeoIP source is derived from target when source field 'ip' is a subfield of 'target' should set other subfields of 'target' properly

Randomized with seed 14034

Thanks everyone for all the help!

Jesse.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.