Unable to send data from metricbeat to logstash

Hello ,

Can someone help in this,I'm unable to send data from metricbeat to logstash.
Below are the configurations:

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.kibana:
  host: http://abc06appl002.dev.ppdm01.group.w:5601
output.logstash:
   hosts: ["abc06appl002.dev.ppdm01.group.w:5044"]
processors:
- add_fields:
    fields:
        configfile: ${DM_SHARED}/MIS/globalconfiguration.properties
- rename:
    fields:
    - from: agent.hostname
      to: host.name
    ignore_missing: true
    fail_on_error: false
- drop_fields:
    fields:
 #  - host.mac
 #  - system.process.cmdline
    - ecs
    - agent.ephemeral_id
    - agent.type
    - agent.version
    - agent.hostname
    - event
monitoring.enabled: true
monitoring.elasticsearch.hosts: true
#logging.level: debug

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}



output {

    stdout {}
}

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:systemmetricbeat, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (PluginLoadingError) Couldn't find any input plugin named 'beats'. Are you sure this is correct? Trying to load the beats input plugin resulted in this error: Problems loading the requested plugin named beats of type input. Error: RuntimeError \n\n\tyou might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars with lock_jars command\n\nno such file to load -- org/apache/logging/log4j/log4j-api/2.11.1/log4j-api-2.11.1 (LoadError)", :backtrace=>["org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:119)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:82)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "l.app.CIS.logstash.logstash_minus_7_dot_9_dot_1.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/l/app/CIS/logstash/logstash-7.9.1/logstash-core/lib/logstash/java_pipeline.rb:44)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "l.app.CIS.logstash.logstash_minus_7_dot_9_dot_1.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/l/app/CIS/logstash/logstash-7.9.1/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "l.app.CIS.logstash.logstash_minus_7_dot_9_dot_1.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$VARARGS(/l/app/CIS/logstash/logstash-7.9.1/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "l.app.CIS.logstash.logstash_minus_7_dot_9_dot_1.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/l/app/CIS/logstash/logstash-7.9.1/logstash-core/lib/logstash/agent.rb:357)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.lang.Thread.run(Thread.java:748)"]}
warning: thread "Converge PipelineAction::Create" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle Java::JavaLang::IllegalStateException for PipelineAction::Create<systemmetricbeat>
create at org/logstash/execution/ConvergeResultExt.java:129
add at org/logstash/execution/ConvergeResultExt.java:57
converge_state at /l/app/CIS/logstash/logstash-7.9.1/logstash-core/lib/logstash/agent.rb:370
[2022-04-04T15:27:16,324][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle Java::JavaLang::IllegalStateException for PipelineAction::Create<systemmetricbeat>>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:129:in create'", "org/logstash/execution/ConvergeResultExt.java:57:in add'", "/l/app/CIS/logstash/logstash-7.9.1/logstash-core/lib/logstash/agent.rb:370:in `block in converge_state'"]}
[2022-04-04T15:27:16,370][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit


  - pipeline.id: tomcatmonitor
    queue.type: persisted
    path.config: "/l/app/MIS/logstash/logstash-7.9.1/scripts/tomcat/monitortomcat.cfg"
  - pipeline.id: oracledb
    queue.type: persisted
    path.config: "/l/app/MIS/logstash/logstash-7.9.1/bin/cis-sql.cfg"
  - pipeline.id: systemmetricbeat
    queue.type: persisted
    path.config: "/l/app/MIS/logstash/logstash-7.9.1/bin/systemMetricbeat.cfg"



metricbeat.logs

Connecting to backoff(async(tcp://td06appl002.dev.kpdm01.group.wg:5044))
2022-04-04T15:25:52.053+0200	INFO	[publisher]	pipeline/retry.go:219	retryer: send unwait signal to consumer
2022-04-04T15:25:52.053+0200	INFO	[publisher]	pipeline/retry.go:223	  done
2022-04-04T15:25:53.792+0200	ERROR	[publisher_pipeline_output]	pipeline/output.go:154	Failed to connect to backoff(async(tcp://td06appl002.dev.ppdm01.group.wg:5044)): dial tcp 192.168.68.65:5044: connect: connection refused

Can someone plz help in this,stuck since a week on this,beats,logstash,kibana,elastic..all are 7.9.1 version.

Many thanks,

I cannot say why that error is occurring, but log4j2 2.11.1 has remote code execution vulnerabilities and logstash 7.9.1 is beyond end-of-support. I suggest you upgrade.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.