Exception=>"NoMethodError", :message=>"undefined method `unlock' for nil:NilClass"

[Marcos_OS (Marcos Oliveira Soares)]

Could someone help me with the following situation:

When I have the filter below, I have the expected result

filter {
  ruby {
	code => '
	  fieldname = event.get("type") + "datref"
	  event.set("datref", event.get(fieldname))
	'
  } 
}

When I add a new condition, I do not have the same result, I'm getting the error below

filter {
  if "jdbc" in [tags] {
    mutate {
      convert => {
        "id_sync" => "text"
      }
    }
    if ["aut", "log", "lan", "ltr"] in [tags] {
      ruby {
        code => '
          fieldname = event.get("type") + "datref"
          event.set("datref", event.get(fieldname))
        '
      }
    }
  }
}

ERROR_LOG:

[2019-01-23T15:02:20,176][ERROR][logstash.agent] 
	Failed to execute action {:action=>LogStash::PipelineAction::Stop/pipeline_id:basa-banklink, :exception=>"NoMethodError", 
	:message=>"undefined method `unlock' for nil:NilClass", 
	:backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/plugin_mixins/jdbc/jdbc.rb:213:in 
	`close_jdbc_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/inputs/jdbc.rb:268:in `stop'", 
	"/usr/share/logstash/logstash-core/lib/logstash/inputs/base.rb:87:in `do_stop'", "org/jruby/RubyArray.java:1734:in `each'", 
	"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:464:in `stop_inputs'", 
	"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:447:in `shutdown'", 
	"/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/stop.rb:14:in 
	`block in execute'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/stop.rb:13:in `execute'",
	"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:317:in `block in converge_state'"]}

[Badger]

I don't think it is related to the ruby filter. A jdbc input filter is shutting down, and it is trying to unlock the connection. However the @connection_lock variable in nil. Hard to see how that could be. Can you post the complete log file (or a link to it)?

[Marcos_OS (Marcos Oliveira Soares)]

Thanks for the feedback.

I put the configuration files and log file at:

[Badger]

That log file does not contain that error. Please post a link to one that does.

Your configuration has 8 jdbc filters. My guess is that something is causing one of them to be closed or restarted before it has been initialialized. In fact it fails to load the pipeline because of

[ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"basa-banklink", :plugin=>"#<LogStash::FilterDelegator:0x4c87dc67 @metric_events_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @metric_events_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @metric_events_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration_in_millis value:0, @id="896d1d111b0ae51e2f2b01f7019e66b763cacdce97dfeb0b072b90fd00d33dea", @klass=LogStash::Filters::Mutate, @metric_events=#LogStash::Instrument::NamespacedMetric:0x6e1186f9, @filter=<LogStash::Filters::Mutate convert=>{"id_sync"=>"text"}, id=>"896d1d111b0ae51e2f2b01f7019e66b763cacdce97dfeb0b072b90fd00d33dea", enable_metric=>true, periodic_flush=>false>>", :error=>"translation missing: en.logstash.agent.configuration.invalid_plugin_register", :thread=>"#<Thread:0xe83bbea@/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:51 run>"}

I wonder if a pipeline restart after that would cause the problem.

[Marcos_OS (Marcos Oliveira Soares)]

Sorry for the previous log.

Here is the log containing the error: new_error_log.log at the same address previously reported.

I reduced the configuration, I commented the pipeline.id: default

What I could not understand is that if I leave only the ruby flag code inside the filter, logstash starts and works without problem.

[Badger]

@Marcos_OS

mutate { convert => { "id_sync" => "text" } }

What an excellent bug! text is not valid there. You want to do a convert to 'string'. Because the mutate filter cannot initialize the rest of the pipeline configuration does not happen and it never initializes the 8 jdbc inputs. So when the config is re-read causing a pipeline restart or in the case of a shutdown you get an error referencing the unlock.

[Marcos_OS (Marcos Oliveira Soares)]

Thanks for the feedback. I was writing the text below when you answered me.

I changed the amount of JDBC entries from 8 to 1, but I got the same error.

I believe it's something with the mutate filter, because if I remove the block exactly:

     mutate {
       convert => {
         "id_sync" => "text"
       }
     }
Logstash starts successfully.

So I'll rethink how I'll do this conversion out of logstash. Would you have any suggestions?

Do I need to open the bug somewhere or do you already do this just by having the post?

[Badger]

@Marcos_OS

mutate { convert => { "id_sync" => "text" } }

 mutate {
   convert => {
     "id_sync" => "string"
   }
 }

[Marcos_OS (Marcos Oliveira Soares)]

@Badger

string

That I hesitate, I was paying attention to the types already indexed in elasticsearch and not to the types described in the address: Mutate filter plugin | Logstash Reference [8.11] | Elastic

Sorry for the unnecessary post.