Logstash not starting when using "codec => multiline"

Elastic Stack Logstash
1

My objective is to parse a multiline string. For that, I tried to use “codec => multiline”. But whenever I try to start logstash using “codec => multiline”, it fails to start.

But it starts successfully without the “codec => multiline” in configuration file .

The error is as shown below.
D:\Tools\ELK\logstash-7.8.0>bin\logstash -f config\logstash0722c2.conf
Picked up JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
Sending Logstash logs to D:/Tools/ELK/logstash-7.8.0/logs which is now configured via log4j2.properties
[2020-07-24T09:36:03,958][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-07-24T09:36:04,075][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.8.0", "jruby.version"=>"jruby 9.2.11.1 (2.5.7) 2020-03-25 b1f55b1a40 Java HotSpot(TM) 64-Bit Server VM 25.231-b11 on 1.8.0_231-b11 +indy +jit [mswin32-x86_64]"}
[2020-07-24T09:36:06,502][INFO ][org.reflections.Reflections] Reflections took 40 ms to scan 1 urls, producing 21 keys and 41 values
[2020-07-24T09:36:06,907][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (TypeError) Java type is not serializable, cannot be marshaled class org.logstash.config.ir.imperative.PluginStatement", :backtrace=>["org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:119)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:80)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "D_3a.Tools.ELK.logstash_minus_7_dot_8_dot_0.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(D:/Tools/ELK/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:43)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:82)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "D_3a_.Tools.ELK.logstash_minus_7_dot_8_dot_0.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(D:/Tools/ELK/logstash-7.8.0/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "D_3a_.Tools.ELK.logstash_minus_7_dot_8_dot_0.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$VARARGS(D:/Tools/ELK/logstash-7.8.0/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:82)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "D_3a_.Tools.ELK.logstash_minus_7_dot_8_dot_0.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(D:/Tools/ELK/logstash-7.8.0/logstash-core/lib/logstash/agent.rb:342)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.lang.Thread.run(Thread.java:748)"]}
warning: thread "Converge PipelineAction::Create" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle Java::JavaLang::IllegalStateException for PipelineAction::Create<main>
create at org/logstash/execution/ConvergeResultExt.java:129
add at org/logstash/execution/ConvergeResultExt.java:57
converge_state at D:/Tools/ELK/logstash-7.8.0/logstash-core/lib/logstash/agent.rb:355
[2020-07-24T09:36:06,918][ERROR][logstash.agent ] An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle Java::JavaLang::IllegalStateException for PipelineAction::Create<main>", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:129:in create'", "org/logstash/execution/ConvergeResultExt.java:57:in add'", "D:/Tools/ELK/logstash-7.8.0/logstash-core/lib/logstash/agent.rb:355:in block in converge_state'"]} [2020-07-24T09:36:06,957][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle Java::JavaLang::IllegalStateExceptionforPipelineAction::Create>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:129:in create'", "org/logstash/execution/ConvergeResultExt.java:57:in add'", "D:/Tools/ELK/logstash-7.8.0/logstash-core/lib/logstash/agent.rb:355:in block in converge_state'"]}
[2020-07-24T09:36:06,972][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

As the filter multiline is deprecated I am trying to use codec => multiline for parsing log details spread over multiline.

2

What does your input configuration look like?

3

The input file looks like below:
# Sample Logstash configuration for creating a simple output from log file

input {
     file {
       type => "reportpublisherlog"
       tags => "reportpublisher"
       path => [ "D:/Tools/ELK/logstash-7.8.0/data/input/inlog1.log" ]
       codec => plain { 
			charset => "ISO-8859-1" 
	   }
	   codec => multiline {
			pattern => "^\s"
			what => "previous"
	   }
       #for debugging
       start_position => "beginning"
       sincedb_path => "NUL" #Setting sincedb_path => "NUL" (in windows) OR "/dev/null" in linux causes logStash to read the old lines (before starting log stash as well as the new lines in input file
	   
     }
}
filter {
	 grok {
			  patterns_dir => ["D:/Tools/ELK/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/logstash-patterns-core-4.1.2/patterns"]
              match => { "message" => "%{GETDATA:getdata}" }
			  match => { "message" => "%{TRANSFORM:transform}" }
      }
	  mutate {
		gsub => ["message", "[\r]$", ""]
	  }
}
output {
	if "_grokparsefailure" in [tags] {
		
	}
	else {
	   csv {
		  path => "D:/Tools/ELK/logstash-7.8.0/data/output/outlog1.log"
		  fields => ["message"]
	   }
    }
}
4

You cannot have both codec => plain and codec => multiline on the same input.

1 Like
5

Thank You for your reply. This worked. means when I had only "codec => multiline" in my config file things worked as expected.
But wanted to check if there's any way to have multiple codecs in the config file like I had "codec => plain" along with "codec => multiline". I believe I'd require the "codec => plain" also as part of the config to handle some specific cases.
I might need to have multiple codecs for parsing the input log file.

6

Not that I know of.

7

okay. Thank You!

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.