Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"NoMethodError", :message=>"undefined method `close' for nil:NilClass"

Elastic Stack Logstash
1 
 [2022-11-17T09:26:38,925][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.8.23"}
[2022-11-17T09:26:42,617][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"NoMethodError", :message=>"undefined method `close' for nil:NilClass", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:83:in `add_patterns_from_file'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-multiline-3.0.11/lib/logstash/codecs/multiline.rb:162:in `block in register'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-multiline-3.0.11/lib/logstash/codecs/multiline.rb:160:in `block in register'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-multiline-3.0.11/lib/logstash/codecs/multiline.rb:155:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/codecs/base.rb:18:in `initialize'", "org/logstash/plugins/PluginFactoryExt.java:255:in `plugin'", "org/logstash/plugins/PluginFactoryExt.java:184:in `plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:71:in `plugin'", "(eval):8:in `initialize'", "org/jruby/RubyKernel.java:1061:in `eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:43:in `block in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:96:in `block in exclusive'", "org/jruby/ext/thread/Mutex.java:165:in `synchronize'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:96:in `exclusive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:39:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:334:in `block in converge_state'"]}

Input file is:

input {
  file {
    path => "/var/log/aerospike/aerospike.log"
    type => "asd"
    start_position => "end" # Default = end
    codec => multiline {
         patterns_dir=> ["/etc/logstash/multi_pattern"]
         pattern => "%{HISTOGRAM}|%{AS_HEADER}stacktrace: %{AS_REST:stacktrace}"
         what => "previous"
    }
  }
}

filter {
    if [type] == "asd" {
        grok {
                match => {
                        "path" => ["/logs/%{GREEDYDATA:logfile}"]
                }
        }
        if "multiline" in [tags] {
                grok {
                        patterns_dir => ["/etc/logstash/patterns"]
                        match => {
                                "message" => ["%{HIST_START}(%{SPACE})+%{AS_REST:histograms}"]
                        }
                }
                # histograms are saved into the 'histograms' field from the previous grok
                mutate {
                        gsub => [
                                # remove everything before the double spaces
                                "histograms", ".*(\s{2})",""
                        ]
                }
                kv{
                        value_split => ": "
                        field_split => "() "
                        target => "histogram"
                        source => "histograms"
                        remove_field => [ "histograms" ]
                }
                ruby {
                     code => "
                         kv = event.get('histogram')
                         kv.to_hash.keys.each { |k|
                             kv[k] = kv[k].to_i
                         }
                         event.set('histogram',kv)
                         "
                 }
        } else {
                grok {
                        patterns_dir => ["/etc/logstash/patterns"]
                        match => {
                                "message" => ["%{AS_NS}","%{AS_TICK}","%{AS_NSUP}","%{AS_XDR}","%{AS_SINDEX}","%{AS_ERR}","%{AS_LOG}"]
                        }
                }
        }
        date {
           match => [ "logdate", "MMM dd YYYY HH:mm:ss ZZZ", "MMM d YYYY HH:mm:ss ZZZ"]
        }
    }
}

output {
  elasticsearch {
   hosts => "xxxxxx"
   index => 'aerospike-%{+YYYY.MM.dd}'
  }
}
2

You need to provide context about your issue, just pasting the error as the title won't help.

What is the issue? What you are trying to do?

3

When I am trying to start my logstash service using: systemctl start logstash.service
This particular ERROR is thrown:

2022-11-17T09:26:42,617][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"NoMethodError", :message=>"undefined method `close' for nil:NilClass".

the service was running fine when running it using tarball and using this command:

/opt/logstash-6.8.0/bin/logstash -f /opt/logstash-6.8.0/config/aerospike_logstash.conf --path.settings /opt/logstash-6.8.0/config/ &

Now, Installed logstash as a service. The location of config file is:

/etc/logstash/conf.d/aerospike_logstash.conf

4

This is a known issue. If the patterns file is not readable then grok throws that exception.

5

Thanks for the reply, is there any workaround for this?
It was working like a charm when i used it with tarball but can’t implement that as it crashes when a server restarts

6

Make sure that all the files in patterns_dir are accessible by the user that the service runs as.

7

I already gave 777 permission to patterns directory and files. What else i can do?

8

Take a look at this thread for ways to verify that the user has access.

9

I am running the service as root user
This is the output I am getting. Is this correct?
Screenshot from 2022-11-18 12-03-27

10

Getting this error after changing the permissions

2022-11-18T06:59:25,048][ERROR][logstash.pipeline        ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::File start_position=>\"end\", path=>[\"/var/log/aerospike/aerospike.log\"], codec=><LogStash::Codecs::Multiline pattern=>\"%{HISTOGRAM}|%{AS_HEADER}stacktrace: %{AS_REST:stacktrace}\", what=>\"previous\", id=>\"fd4f96e4-6c23-4b5b-b2bc-6b0530f7812d\", patterns_dir=>[\"/etc/logstash/multi_pattern\"], enable_metric=>true, negate=>false, charset=>\"UTF-8\", multiline_tag=>\"multiline\", max_lines=>500, max_bytes=>10485760>, id=>\"491d5547723009700acc9d553580bf8a05151d959ed2aba2b5c98c4318d6eeae\", type=>\"asd\", enable_metric=>true, stat_interval=>1.0, discover_interval=>15, sincedb_write_interval=>15.0, delimiter=>\"\\n\", close_older=>3600.0, mode=>\"tail\", file_completed_action=>\"delete\", sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_by=>\"last_modified\", file_sort_direction=>\"asc\", exit_after_read=>false, check_archive_validity=>false>", :error=>"Permission denied - /var/lib/logstash/plugins/inputs", :thread=>"#<Thread:0x51cb9dca run>"}
[2022-11-18T06:59:25,630][ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<Errno::EACCES: Permission denied - /var/lib/logstash/plugins/inputs>, :backtrace=>["org/jruby/RubyDir.java:590:in `mkdir'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:235:in `fu_mkdir'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:213:in `block in mkdir_p'", "org/jruby/RubyArray.java:1876:in `reverse_each'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:211:in `block in mkdir_p'", "org/jruby/RubyArray.java:1792:in `each'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:196:in `mkdir_p'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/pathname.rb:579:in `mkpath'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.18/lib/logstash/inputs/file.rb:405:in `block in build_sincedb_base_from_settings'", "org/jruby/RubyKernel.java:1885:in `tap'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.18/lib/logstash/inputs/file.rb:403:in `build_sincedb_base_from_settings'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.18/lib/logstash/inputs/file.rb:288:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:259:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:270:in `block in register_plugins'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:270:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:413:in `start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:311:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:217:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:176:in `block in start'"], :thread=>"#<Thread:0x51cb9dca run>"}
[2022-11-18T06:59:25,644][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
11

The whole issue was regarding the permissions, logstash user wasn't having enough permissions.
Added it to the root group and it worked like a charm.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.