Logstash8.12.2 restart many times may encounter plugin load errors 'cabin/metrics','cabin/log/logger', "logstash::codecs::multiline::grok"

Elastic Stack Elasticsearch
1

I upgrade the logstash from 7.16.3 to 8.12.2 and it can start up successfully after the configuration file adjustment, but if I restart the logstash service time from times it may encounter above plugin load errors. Can anybody help me on this? Is this caused by the unstable of logstash8.12.2 or something else?

2

Welcome to the community
Have you changed .conf or logstash.yml
Have you used .tar version or installation package .rpm?
Can you provide more details?

LS 8.12.2 works fine.

6

Please do not post pictures.

It might be related to JDK, but not sure. Please provide more details.

10

There is something wrong with a grok pattern separated in a file. Either path or permissions have an issue.

11

Q:

  1. if it's related to the grok pattern then why when we use ls 7.16.3 it works well in our production environment ?

  2. 2.if it's related to path then why some times the startup will succeed without any error log

    1. same to above two questions
12

by the way, the user and the group is default to logstash

13
  1. No sure, however there is the trace in the log: add_patterns_from_file.
  2. Restart after some time might be because:
  • damaged installation package - in that case, reinstall
  • memory or some other nonELK issues
    Check journalctl, is there some issues
  1. Yes can be, because some params has been changed. You have set:
output{
 if [myTYpe] in ["my-type"] {
    elasticsearch {
    hosts => ["https://localhost:9200"] # has the cert issued on "localhost" or hostname, must be correct
    ssl_enabled => true
    ssl_certificate_authorities => "/etc/logstash/config/certs/ca.crt"  # not as you set ssl_certificate_authoritiesE 
    ssl_verification_mode => "full" # try also with none
    }
 }
}
16

It's a waste of time to guess.

Please provide a sample of message and your .conf file.

18

I checked this from logstash github found this part of module is comming from ruby ... maybe there's something wrong with the ruby version ?

require "cabin/namespace"
require "cabin/timer"

module Cabin::Mixins::Timer

Start timing something.

Returns an instance of Cabin::Timer bound to this Cabin::Channel.

To stop the timer and immediately emit the result to this channel, invoke

the Cabin::Timer#stop method.

def time(data, &block)
# TODO(sissel): need to refactor string->hash shoving.
data = dataify(data)

timer = Cabin::Timer.new do |duration|
  data[:duration] = duration
  publish(data)
end

if block_given?
  block.call
  return timer.stop
else
  return timer
end

end # def time
end # module Cabin::Mixins::Timer

19

there must be some plugin try to require this module but failed

20

that cabin/mixins/timer plugin load error might happened in this part

codec =>multiline {

pattern => "^%{TIMESTAMP_ISO8601}"
negate => true
what => "previous"

}

21

You need to provide a sample message and your logstash configuration so people can try to replicate your error, without this information it is just guessing and there is not much to do to help you.

1 Like
22

There is something wrong with a grok pattern separated in a file.

23

I comment out other pipeline and start one by one there's no error no matter how many times I restart but if I include a special pipeline the other pipeline will have the plugin load error, when I comment out that special pipeline it all works fine, I'm just confused with this why that special pipeline start speparately is ok but if start at the same time with other pipeline will cause other pipeline failed to startup

24

Thanks everyone problem resolved, it's due to one of my pipleline conf timestamp match error

25

o gosh I really don't know why mutate rename would cause string index out of range -1 exception ......

26

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.