Logstash isn't starting. Getting a FATAL error

I am new to ELK stack. I am able to start Elasticsearch and Kibana but unable to start Logstash using command logstash -f logstash-studio.conf

Version 7.14
Java Version 1.8

Here are the logs:-

Using JAVA_HOME defined java: C:\Program Files\Java\jdk1.8.0_333
WARNING, using JAVA_HOME while Logstash distribution comes with a bundled JDK
C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/vendor/bundle/jruby/2.5.0/gems/bundler-1.17.3/lib/bundler/rubygems_integration.rb:200: warning: constant Gem::ConfigMap is deprecated
**[FATAL] 2022-07-06 12:18:09.416 [main] runner - An unexpected error occurred! {:error=>#<TypeError: no implicit conversion of nil into String>, :backtrace=>["org/jruby/RubyFileTest.java:95:in `directory?'", "org/jruby/RubyFileTest.java:87:in `directory?'",** "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/settings.rb:551:in `block in value'", "org/jruby/RubyKernel.java:1891:in `tap'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/settings.rb:550:in `value'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/settings.rb:116:in `get_value'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/environment.rb:115:in `block in LogStash'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/settings.rb:186:in `block in post_process'", "org/jruby/RubyArray.java:1820:in `each'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/settings.rb:185:in `post_process'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/util/settings_helper.rb:43:in `post_process'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/runner.rb:278:in `execute'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/runner.rb:274:in `run'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "C:\\ELK_Stack\\logstash-7.14.0-windows-x86_64\\logstash-7.14.0\\lib\\bootstrap\\environment.rb:88:in `<main>'"]}
**[FATAL] 2022-07-06 12:18:09.418 [main] Logstash - Logstash stopped processing because of an error: (SystemExit) exit**
**org.jruby.exceptions.SystemExit: (SystemExit) exit**
**        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.19.0.jar:?]**
**        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.19.0.jar:?]**
**        at C_3a_.ELK_Stack.logstash_minus_7_dot_14_dot_0_minus_windows_minus_x86_64.logstash_minus_7_dot_14_dot_0.lib.bootstrap.environment.<main>(C:\ELK_Stack\logstash-7.14.0-windows-x86_64\logstash-7.14.0\lib\bootstrap\environment.rb:89) ~[?:?]**

Here are the contents of logstash-studio.conf file which I have kept in bin folder.

#Sample Logstash configuration
#Beats => Logstash => Elasticsearch pipeline
input {
	tcp {
		port => 4560
		codec => json
	}
}
filter{
	date {
		match => [ "timeMillis", "UNIX MS"]
	}
}
output {
	elasticsearch {
		hosts => ["localhost:9200"]
		index => "mule-logs"
	}
}

Not sure what your first error is ... but if you are really doing

#Beats => Logstash => Elasticsearch pipeline
then your input / output should look like the following especially if you are using a module.

################################################
# beats->logstash->es default config.
################################################
input {
  beats {
    port => 5044
  }
}

output {
  if [@metadata][pipeline] {
    elasticsearch {
      hosts => "http://localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      pipeline => "%{[@metadata][pipeline]}" 
      user => "elastic"
      password => "secret"
    }
  } else {
    elasticsearch {
      hosts => "http://localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      user => "elastic"
      password => "secret"
    }
  }
}

Your field timeMillis doesn't exist at all or sometimes a value is missing.
AFAIK, the tcp plugin receive data as a message which contains fields, not pure fields.
Use debugger to discover your error.

output {
    stdout { codec => rubydebug{}  }
}

Please ignore the below comment from logstash-studio.conf file
#Beats => Logstash => Elasticsearch pipeline

I just realized that entry of path.data in logstash.yml was empty. I updated it to path.data: "/data". Getting the below error now.

Using JAVA_HOME defined java: C:\Program Files\Java\jdk1.8.0_333
WARNING, using JAVA_HOME while Logstash distribution comes with a bundled JDK
C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/vendor/bundle/jruby/2.5.0/gems/bundler-1.17.3/lib/bundler/rubygems_integration.rb:200: warning: constant Gem::ConfigMap is deprecated
Sending Logstash logs to C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logs which is now configured via log4j2.properties
[2022-07-06T12:58:40,573][INFO ][logstash.runner          ] Log4j configuration path used is: C:\ELK_Stack\logstash-7.14.0-windows-x86_64\logstash-7.14.0\config\log4j2.properties
[2022-07-06T12:58:40,573][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.14.0", "jruby.version"=>"jruby 9.2.19.0 (2.5.8) 2021-06-15 55810c552b Java HotSpot(TM) 64-Bit Server VM 25.333-b02 on 1.8.0_333-b02 +indy +jit [mswin32-x86_64]"}
[2022-07-06T12:58:40,605][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/data/queue"}
[2022-07-06T12:58:40,605][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/data/dead_letter_queue"}
[2022-07-06T12:58:40,673][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-07-06T12:58:40,705][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"830564a0-cc98-4678-99f1-1db1e28eccb7", :path=>"/data/uuid"}
[2022-07-06T12:58:42,920][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2022-07-06T12:58:43,375][INFO ][org.reflections.Reflections] Reflections took 54 ms to scan 1 urls, producing 120 keys and 417 values
[2022-07-06T12:58:44,160][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: Illegal pattern component: U", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:119)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:86)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "C_3a_.ELK_Stack.logstash_minus_7_dot_14_dot_0_minus_windows_minus_x86_64.logstash_minus_7_dot_14_dot_0.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/java_pipeline.rb:47)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:333)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:87)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "C_3a_.ELK_Stack.logstash_minus_7_dot_14_dot_0_minus_windows_minus_x86_64.logstash_minus_7_dot_14_dot_0.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "C_3a_.ELK_Stack.logstash_minus_7_dot_14_dot_0_minus_windows_minus_x86_64.logstash_minus_7_dot_14_dot_0.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "C_3a_.ELK_Stack.logstash_minus_7_dot_14_dot_0_minus_windows_minus_x86_64.logstash_minus_7_dot_14_dot_0.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/agent.rb:391)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.lang.Thread.run(Thread.java:750)"]}
warning: thread "Converge PipelineAction::Create<main>" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`
          create at org/logstash/execution/ConvergeResultExt.java:135
             add at org/logstash/execution/ConvergeResultExt.java:60
  converge_state at C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/agent.rb:404
[2022-07-06T12:58:44,176][ERROR][logstash.agent           ] An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`"}
[2022-07-06T12:58:44,182][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:135:in `create'", "org/logstash/execution/ConvergeResultExt.java:60:in `add'", "C:/ELK_Stack/logstash-7.14.0-windows-x86_64/logstash-7.14.0/logstash-core/lib/logstash/agent.rb:404:in `block in converge_state'"]}
[2022-07-06T12:58:44,198][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.19.0.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.19.0.jar:?]
        at C_3a_.ELK_Stack.logstash_minus_7_dot_14_dot_0_minus_windows_minus_x86_64.logstash_minus_7_dot_14_dot_0.lib.bootstrap.environment.<main>(C:\ELK_Stack\logstash-7.14.0-windows-x86_64\logstash-7.14.0\lib\bootstrap\environment.rb:89) ~[?:?]

What is the exact command you are running? you keep leaving that out ... that is always helpful for us...

2nd I agree with @Rios I don't think your input is not working the way you think it is..

I would try this... to see what you are really getting

input {
  tcp {
    port => 4560
  }
}

output {
  stdout { codec => json }
}

Then start building your filters etc.

1 Like

Yes, as @stephenb said, keep it simple at the begging to see your data format.
If you are doing for the first time, remove codecs, clean the filter are, set output only to stdout with ruby debug or plain text, then add parser, codec and type conversions.
When everything is set, add elastic output .

1 Like

Thanks for the inputs @Rios and @stephenb . I am trying to do a POC where logs from my mulesoft application will be sent to Logstash -> Elasticsearch -> Kibana.
I made changes to my logstash-studio.conf file as per your inputs. It looks like this now

input {
	tcp {
		port => 4560
		codec => json
	}
}

output {
	elasticsearch {
		hosts => ["localhost:9200"]
		index => "mule-logs"
	}
}

Command - logstash -f logstash-studio.conf

No error now. Will check the filter part

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.