The whole log of openedr is inside the message feild in kibana. Urgent!

Elastic Stack Beats
1

Hi, my openedr application is in windows i have downloaded filebeat there and configure it to send logs to logstash so that i can view in kibana and yes it done successfully. I'm able to see logs in kibana>discover,
But the whole log entry is inside the message field it was not parsing it correctly
i have created a filebeat* as index pattern and dataview as openedr_logs because it creating new index pattern with the name filebeat along with the date everyday, it has all the fields which the log entry required but still those fields are empty the whole log entry is inside the message field and also in event.original field

image
image1846×763 227 KB

my logstash conf is:

Beats -> Logstash -> Elasticsearch pipeline.

input {
beats {
port => 5044
}
}

filter {
json {
source => "event.original"
target => "parsed_event"
}

if [parsed_event] {
mutate {
add_field => { "customer_id" => "%{[parsed_event][customerId]}" }
add_field => { "device_name" => "%{[parsed_event][deviceName]}" }
add_field => { "base_event_type" => "%{[parsed_event][baseEventType]}" }
add_field => { "base_type" => "%{[parsed_event][baseType]}" }
add_field => { "session_user" => "%{[parsed_event][sessionUser]}" }
add_field => { "event_type" => "%{[parsed_event][eventType]}" }
add_field => { "version" => "%{[parsed_event][version]}" }
# Add similar lines for other desired fields
}
}
}

output {
elasticsearch {
hosts => ["https://ip-addr:9200"]
user => "elastic"
password => "password"
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
ssl_verification_mode => "full"
ssl_certificate_authorities => "/etc/elasticsearch/certs/http_ca.crt"
ssl_enabled => true
}
}
even though in conf i mention as index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" the logs are getting stored in filebeat* index. If i change index pattern as filebeat* its not taking.

please suggest!

2

Should be:

json {
source => "[event][original]"
target => "parsed_event"
}
3

thanks for the reply
if i add the below one

json {
source => "[event][original]"
target => "parsed_event"
}

i'm not able to see the any logs in kibana.

image
image1845×865 82.1 KB

if i add the below one i'm able to see the logs but whole log entry inside event.original and in message.
json {
source => "event.original"
target => "parsed_event"
}

image
image1845×865 152 KB

image
image1845×865 241 KB

even i have tried with
filter {
json {
source => "message"
target => "parsed_event"
}
same im not able to see any logs in kibana.

4

You cen use source => "[event][original]" or source => "message". Parsed data will be in [parsed_event][fieldname]. If you don't set, fields will on the root.

If this setting is omitted, the JSON data will be stored at the root (top level) of the event.

Take longer range and check index name. Your data should be in filebeat_8.12.2(by default) or openEDR_logs if you set this, depend on FB settings, in filebeat.yml.
Also, might be some delay take a wider range.

To see your data structure, add in the output part, temporarily :
stdout { codec => rubydebug{} }

5

source => "[event][original]" or source => "message" this both is not working, only source => "event.original" is working this stores the whole log entry inside the event.original field. I'm not able to see any field called [parsed_event]. Actually it have to parse the log in individual fields right it not doing that.

And i'm using the default index pattern only as it is creating daily one index pattern so i have set it filebeat*

image
image1845×865 120 KB

In filebeat.yml file i'm using output as logstash i commented elasticsearch output, filebeat and elk is in different machine

6

Can you show the message field -data when arrived in LS? In the text format, not the image.

7

message

{"baseEventType":1,"baseType":1,"childProcess":{"cmdLine":"C:\Windows\system32\svchost.exe -k netsvcs -p -s gpsvc","creationTime":1714556553033,"elevationType":1,"flsVerdict":3,"id":5349677356,"imageHash":"445f5f38396f0e3ee50a1d8","imagePath":"C:\Windows\System32\svchost.exe","pid":14048,"scriptContent":"","verdict":1},"customerId":"","deviceName":"IZDT-6","endpointId":"","eventType":null,"processes":[{"creationTime":171430626,"flsVerdict":3,"id":13324682947529074784,"imageHash":"2dfb00cd9a44a8a016452e1df01c0cec51870407","imagePath":"C:\Windows\System32\wininit.exe","pid":784,"userName":"SYSTEM@NT AUTHORITY","verdict":1},{"creationTime":1714392570787,"flsVerdict":3,"id":763785080847475,"imageHash":"e5704d8e560122c2a23de8912ae63b213d67c860","imagePath":"C:\Windows\System32\services.exe","pid":856,"userName":"SYS@NT AUTHORITY","verdict":1}],"sessionUser":"SYS@NT AUTHORITY","time":17145085,"type":"RP1.1","version":"1.1"}

8

If I am not wrong, the json plugin have some "strange features" with the backslashes. So, replace \ with / then return back :slight_smile:
This works:

 input {
     generator {
       message => '{"baseEventType":1,"baseType":1,"childProcess":{"cmdLine":"C:\Windows\system32\svchost.exe -k netsvcs -p -s gpsvc","creationTime":1714556553033,"elevationType":1,"flsVerdict":3,"id":5349677356,"imageHash":"445f5f38396f0e3ee50a1d8","imagePath":"C:\Windows\System32\svchost.exe","pid":14048,"scriptContent":"","verdict":1},"customerId":"","deviceName":"IZDT-6","endpointId":"","eventType":null,"processes":[{"creationTime":171430626,"flsVerdict":3,"id":13324682947529074784,"imageHash":"2dfb00cd9a44a8a016452e1df01c0cec51870407","imagePath":"C:\Windows\System32\wininit.exe","pid":784,"userName":"SYSTEM@NT AUTHORITY","verdict":1},{"creationTime":1714392570787,"flsVerdict":3,"id":763785080847475,"imageHash":"e5704d8e560122c2a23de8912ae63b213d67c860","imagePath":"C:\Windows\System32\services.exe","pid":856,"userName":"SYS@NT AUTHORITY","verdict":1}],"sessionUser":"SYS@NT AUTHORITY","time":17145085,"type":"RP1.1","version":"1.1"}'
	   count => 1
  }
}
filter {
	mutate { gsub => [ "message", "[\\]", "/" ] }
	json{ source=>"message"}
	
	ruby{
    code => ' 
	event.set("[childProcess][cmdLine]", event.get("[childProcess][cmdLine]").gsub("/", "\\\\") ) 
	event.set("[childProcess][imagePath]", event.get("[childProcess][imagePath]").gsub("/", "\\\\") ) 
	event.set("message", event.get("message").gsub("/", "\\\\") ) 
	
    proc = event.get("processes")
    proc.each_with_index do |value, index|
         event.set("[processes][#{index}][imagePath]", event.get("[processes][#{index}][imagePath]").gsub("/", "\\\\"))
    end
	'
  }
}
output {
 stdout { codec => rubydebug{} }
}

Maybe someone has better idea.

9

thank you for spending your precious time on my issues :innocent:
i will try the above one and let you know

10

What is your Logstash version? You didn't say, the version is importanto for the troubleshooting.

My suggestion is to only add the Elasticsearch output after you fixed the parse of your data.

For example, configure a file output to see what logstash is receiving.

Start with a single pipeline without any filters, just the input and output, this way you will know what is the raw message logstash is receiving and the right field name to use.

Use this:

input {
    beats {
        port => 5044
    }
}
output {
    file {
        path => "/tmp/logstash-raw-input.log"
    }
}

Then share some lines of the output file.

1 Like
11

hi, my logstash version is 8.13.2. If i use the below pipeline
input {
beats {
port => 5044
}
}
output {
file {
paht => "/tmp/logstash-raw-input.log"
}
}
im getting the below error while when i run the logstash

cmd: bin/logstash -f /etc/logstash/conf.d/openedr.conf

o/p:
Using bundled JDK: /usr/share/logstash/jdk
/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/java_thread_pool_executor.rb:13: warning: method redefined; discarding old to_int
/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/executor/java_thread_pool_executor.rb:13: warning: method redefined; discarding old to_f
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2024-05-02 11:27:25.886 [main] runner - NOTICE: Running Logstash as superuser is not recommended and won't be allowed in the future. Set 'allow_superuser' to 'false' to avoid startup errors in future releases.
[INFO ] 2024-05-02 11:27:25.894 [main] runner - Starting Logstash {"logstash.version"=>"8.13.2", "jruby.version"=>"jruby 9.4.5.0 (3.1.4) 2023-11-02 1abae2700f OpenJDK 64-Bit Server VM 17.0.10+7 on 17.0.10+7 +indy +jit [x86_64-linux]"}
[INFO ] 2024-05-02 11:27:25.896 [main] runner - JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dlogstash.jackson.stream-read-constraints.max-string-length=200000000, -Dlogstash.jackson.stream-read-constraints.max-number-length=10000, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Dio.netty.allocator.maxOrder=11]
[INFO ] 2024-05-02 11:27:25.899 [main] runner - Jackson default value override logstash.jackson.stream-read-constraints.max-string-length configured to 200000000
[INFO ] 2024-05-02 11:27:25.899 [main] runner - Jackson default value override logstash.jackson.stream-read-constraints.max-number-length configured to 10000
[WARN ] 2024-05-02 11:27:26.055 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2024-05-02 11:27:26.481 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9601, :ssl_enabled=>false}
[INFO ] 2024-05-02 11:27:26.686 [Converge PipelineAction::Create] Reflections - Reflections took 80 ms to scan 1 urls, producing 132 keys and 468 values
[INFO ] 2024-05-02 11:27:26.900 [Converge PipelineAction::Create] jsonlines - ECS compatibility is enabled but target option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the target option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[ERROR] 2024-05-02 11:27:26.901 [Converge PipelineAction::Create] file - Unknown setting 'paht' for file
[ERROR] 2024-05-02 11:27:26.906 [Converge PipelineAction::Create] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.", :backtrace=>["org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:120)", "org.logstash.execution.AbstractPipelineExt.initialize(AbstractPipelineExt.java:186)", "org.logstash.execution.AbstractPipelineExt$INVOKER$i$initialize.call(AbstractPipelineExt$INVOKER$i$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:847)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1319)", "org.jruby.ir.instructions.InstanceSuperInstr.interpret(InstanceSuperInstr.java:139)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:367)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:128)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:115)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:446)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:92)", "org.jruby.RubyClass.newInstance(RubyClass.java:931)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:446)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:92)", "org.jruby.ir.instructions.CallBase.interpret(CallBase.java:548)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:367)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)", "org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:88)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:238)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:225)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:228)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:476)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:293)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:328)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)", "org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:116)", "org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.Block.call(Block.java:144)", "org.jruby.RubyProc.call(RubyProc.java:352)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:111)", "java.base/java.lang.Thread.run(Thread.java:840)"]}
[INFO ] 2024-05-02 11:27:26.918 [LogStash::Runner] runner - Logstash shut down.

and the ```
logstash-raw-input.log

cat /tmp/logstash-raw-input.log
12

hi,
cat /etc/logstash/conf.d/openedr.conf for the below pipeline
input {
beats {
port => 5044
}
generator {
message => '{"baseEventType":1,"baseType":1,"childProcess":{"cmdLine":"C:\Windows\system32\svchost.exe -k netsvcs -p -s gpsvc","creationTime":1714556553033,"elevationType":1,"flsVerdict":3,"id":5349677356,"imageHash":"445f5f38396f0e3ee50a1d8","imagePath":"C:\Windows\System32\svchost.exe","pid":14048,"scriptContent":"","verdict":1},"customerId":"","deviceName":"IZDT-6","endpointId":"","eventType":null,"processes":[{"creationTime":171430626,"flsVerdict":3,"id":13324682947529074784,"imageHash":"2dfb00cd9a44a8a016452e1df01c0cec51870407","imagePath":"C:\Windows\System32\wininit.exe","pid":784,"userName":"SYSTEM@NT AUTHORITY","verdict":1},{"creationTime":1714392570787,"flsVerdict":3,"id":763785080847475,"imageHash":"e5704d8e560122c2a23de8912ae63b213d67c860","imagePath":"C:\Windows\System32\services.exe","pid":856,"userName":"SYS@NT AUTHORITY","verdict":1}],"sessionUser":"SYS@NT AUTHORITY","time":17145085,"type":"RP1.1","version":"1.1"}'
count => 1
}
}
filter {
mutate { gsub => [ "message", "[\]", "/" ] }
json{ source=>"message"}

ruby{
code => ' 
event.set("[childProcess][cmdLine]", event.get("[childProcess][cmdLine]").gsub("/", "\\\\") ) 
event.set("[childProcess][imagePath]", event.get("[childProcess][imagePath]").gsub("/", "\\\\") ) 
event.set("message", event.get("message").gsub("/", "\\\\") ) 

proc = event.get("processes")
proc.each_with_index do |value, index|
     event.set("[processes][#{index}][imagePath]", event.get("[processes][#{index}][imagePath]").gsub("/", "\\\\"))
end
'

}
}
output {
stdout { codec => rubydebug{} }
}

i have added beats port as extra because the logs are in another server.

when i run the logstash it parsing some of the feilds for only 2 to 3 log entry and repeating some error .

o/p:

},
"time" => 17145085,
"version" => "1.1",
"sessionUser" => "SYS@NT AUTHORITY",
"@timestamp" => 2024-05-02T05:59:50.623008193Z,
"customerId" => "",
"message" => "{"baseEventType":1,"baseType":1,"childProcess":{"cmdLine":"C:\Windows\system32\svchost.exe -k netsvcs -p -s gpsvc","creationTime":1714556553033,"elevationType":1,"flsVerdict":3,"id":5349677356,"imageHash":"445f5f38396f0e3ee50a1d8","imagePath":"C:\Windows\System32\svchost.exe","pid":14048,"scriptContent":"","verdict":1},"customerId":"","deviceName":"IZDT-6","endpointId":"","eventType":null,"processes":[{"creationTime":171430626,"flsVerdict":3,"id":13324682947529074784,"imageHash":"2dfb00cd9a44a8a016452e1df01c0cec51870407","imagePath":"C:\Windows\System32\wininit.exe","pid":784,"userName":"SYSTEM@NT AUTHORITY","verdict":1},{"creationTime":1714392570787,"flsVerdict":3,"id":763785080847475,"imageHash":"e5704d8e560122c2a23de8912ae63b213d67c860","imagePath":"C:\Windows\System32\services.exe","pid":856,"userName":"SYS@NT AUTHORITY","verdict":1}],"sessionUser":"SYS@NT AUTHORITY","time":17145085,"type":"RP1.1","version":"1.1"}",
"endpointId" => "",
"childProcess" => {
"cmdLine" => "C:\Windows\system32\svchost.exe -k netsvcs -p -s gpsvc",
"flsVerdict" => 3,
"elevationType" => 1,
"id" => 5349677356,
"imageHash" => "445f5f38396f0e3ee50a1d8",
"creationTime" => 1714556553033,
"imagePath" => "C:\Windows\System32\svchost.exe",
"pid" => 14048,
"scriptContent" => "",
"verdict" => 1
},
"event" => {
"original" => "{"baseEventType":1,"baseType":1,"childProcess":{"cmdLine":"C:\Windows\system32\svchost.exe -k netsvcs -p -s gpsvc","creationTime":1714556553033,"elevationType":1,"flsVerdict":3,"id":5349677356,"imageHash":"445f5f38396f0e3ee50a1d8","imagePath":"C:\Windows\System32\svchost.exe","pid":14048,"scriptContent":"","verdict":1},"customerId":"","deviceName":"IZDT-6","endpointId":"","eventType":null,"processes":[{"creationTime":171430626,"flsVerdict":3,"id":13324682947529074784,"imageHash":"2dfb00cd9a44a8a016452e1df01c0cec51870407","imagePath":"C:\Windows\System32\wininit.exe","pid":784,"userName":"SYSTEM@NT AUTHORITY","verdict":1},{"creationTime":1714392570787,"flsVerdict":3,"id":763785080847475,"imageHash":"e5704d8e560122c2a23de8912ae63b213d67c860","imagePath":"C:\Windows\System32\services.exe","pid":856,"userName":"SYS@NT AUTHORITY","verdict":1}],"sessionUser":"SYS@NT AUTHORITY","time":17145085,"type":"RP1.1","version":"1.1"}",
"sequence" => 0
}
}
[ERROR] 2024-05-02 11:29:56.746 [[main]<beats] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Beats port=>5044, id=>"875fa50123dc0a9dc3730a16fd05809d155d20bb15fdd4a07f21f6c301321990", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_cacf6798-fcf2-493f-9f8f-69736f8ba519", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_enabled=>false, ssl_client_authentication=>"none", ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, ssl_cipher_suites=>["TLS_AES_256_GCM_SHA384", "TLS_AES_128_GCM_SHA256", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], ssl_supported_protocols=>["TLSv1.2", "TLSv1.3"], client_inactivity_timeout=>60, executor_threads=>4, event_loop_threads=>0, add_hostname=>false, tls_min_version=>1, tls_max_version=>1.3>
Error: Address already in use
Exception: Java::JavaNet::BindException
Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:555)
sun.nio.ch.ServerSocketChannelImpl.netBind(sun/nio/ch/ServerSocketChannelImpl.java:337)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:294)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:141)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:562)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:600)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:579)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:260)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.runTask(io/netty/util/concurrent/AbstractEventExecutor.java:173)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:166)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:470)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:569)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:997)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:840)
[INFO ] 2024-05-02 11:29:57.748 [[main]<beats] Server - Starting server on port: 5044
[ERROR] 2024-05-02 11:30:03.766 [[main]<beats] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Beats port=>5044, id=>"875fa50123dc0a9dc3730a16fd05809d155d20bb15fdd4a07f21f6c301321990", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_cacf6798-fcf2-493f-9f8f-69736f8ba519", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_enabled=>false, ssl_client_authentication=>"none", ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, ssl_cipher_suites=>["TLS_AES_256_GCM_SHA384", "TLS_AES_128_GCM_SHA256", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], ssl_supported_protocols=>["TLSv1.2", "TLSv1.3"], client_inactivity_timeout=>60, executor_threads=>4, event_loop_threads=>0, add_hostname=>false, tls_min_version=>1, tls_max_version=>1.3>
Error: Address already in use
Exception: Java::JavaNet::BindException
Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:555)
sun.nio.ch.ServerSocketChannelImpl.netBind(sun/nio/ch/ServerSocketChannelImpl.java:337)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:294)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:141)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:562)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:600)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:579)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:260)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.runTask(io/netty/util/concurrent/AbstractEventExecutor.java:173)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:166)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:470)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:569)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:997)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:840)
[INFO ] 2024-05-02 11:30:04.769 [[main]<beats] Server - Starting server on port: 5044
[ERROR] 2024-05-02 11:30:10.790 [[main]<beats] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Beats port=>5044, id=>"875fa50123dc0a9dc3730a16fd05809d155d20bb15fdd4a07f21f6c301321990", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_cacf6798-fcf2-493f-9f8f-69736f8ba519", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_enabled=>false, ssl_client_authentication=>"none", ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, ssl_cipher_suites=>["TLS_AES_256_GCM_SHA384", "TLS_AES_128_GCM_SHA256", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], ssl_supported_protocols=>["TLSv1.2", "TLSv1.3"], client_inactivity_timeout=>60, executor_threads=>4, event_loop_threads=>0, add_hostname=>false, tls_min_version=>1, tls_max_version=>1.3>
Error: Address already in use
Exception: Java::JavaNet::BindException
Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:555)
sun.nio.ch.ServerSocketChannelImpl.netBind(sun/nio/ch/ServerSocketChannelImpl.java:337)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:294)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:141)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:562)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:600)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:579)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:260)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.runTask(io/netty/util/concurrent/AbstractEventExecutor.java:173)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:166)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:470)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:569)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:997)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:840)
[INFO ] 2024-05-02 11:30:11.793 [[main]<beats] Server - Starting server on port: 5044
[ERROR] 2024-05-02 11:30:17.812 [[main]<beats] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Beats port=>5044, id=>"875fa50123dc0a9dc3730a16fd05809d155d20bb15fdd4a07f21f6c301321990", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_cacf6798-fcf2-493f-9f8f-69736f8ba519", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_enabled=>false, ssl_client_authentication=>"none", ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, ssl_cipher_suites=>["TLS_AES_256_GCM_SHA384", "TLS_AES_128_GCM_SHA256", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], ssl_supported_protocols=>["TLSv1.2", "TLSv1.3"], client_inactivity_timeout=>60, executor_threads=>4, event_loop_threads=>0, add_hostname=>false, tls_min_version=>1, tls_max_version=>1.3>
Error: Address already in use
Exception: Java::JavaNet::BindException
Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:555)
sun.nio.ch.ServerSocketChannelImpl.netBind(sun/nio/ch/ServerSocketChannelImpl.java:337)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:294)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:141)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:562)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:600)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:579)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:260)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.runTask(io/netty/util/concurrent/AbstractEventExecutor.java:173)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:166)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:470)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:569)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:997)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:840)
^C[WARN ] 2024-05-02 11:30:18.260 [SIGINT handler] runner - SIGINT received. Shutting down.
[INFO ] 2024-05-02 11:30:18.815 [[main]<beats] Server - Starting server on port: 5044
^C[FATAL] 2024-05-02 11:30:19.643 [SIGINT handler] runner - SIGINT received. Terminating immediately..
^Croot@freeipa:/usr/share/logstash# cat /etc/logstash/conf.d/openedr.conf

if i mention elasticsearch in the output plugin it not showing any logs in discover>kibana
same output in the console.

13

There was a type, the output needs to be:

output {
    file {
        path => "/tmp/logstash-raw-input.log"
    }
}
14

sorry,

input {
    beats {
        port => 5044
    }
}
output {
    file {
        path => "/tmp/logstash-raw-input.log"
    }
}

this was my conf file and the error is

Pipeline_id:main
  Plugin: <LogStash::Inputs::Beats port=>5044, id=>"bd4b83cf08f23a0dabb7fd175613f2eed2178dc3259f903844c0418c7d6817ef", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_14b864b2-c224-4c38-b2ef-7b916c612351", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_enabled=>false, ssl_client_authentication=>"none", ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, ssl_cipher_suites=>["TLS_AES_256_GCM_SHA384", "TLS_AES_128_GCM_SHA256", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], ssl_supported_protocols=>["TLSv1.2", "TLSv1.3"], client_inactivity_timeout=>60, executor_threads=>4, event_loop_threads=>0, add_hostname=>false, tls_min_version=>1, tls_max_version=>1.3>
  Error: Address already in use
  Exception: Java::JavaNet::BindException
  Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:555)
sun.nio.ch.ServerSocketChannelImpl.netBind(sun/nio/ch/ServerSocketChannelImpl.java:337)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:294)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:141)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:562)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:600)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:579)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:260)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.runTask(io/netty/util/concurrent/AbstractEventExecutor.java:173)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:166)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:470)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:569)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:997)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:840)
[INFO ] 2024-05-02 18:37:26.818 [[main]<beats] Server - Starting server on port: 5044
[ERROR] 2024-05-02 18:37:32.837 [[main]<beats] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Beats port=>5044, id=>"bd4b83cf08f23a0dabb7fd175613f2eed2178dc3259f903844c0418c7d6817ef", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_14b864b2-c224-4c38-b2ef-7b916c612351", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_enabled=>false, ssl_client_authentication=>"none", ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, ssl_cipher_suites=>["TLS_AES_256_GCM_SHA384", "TLS_AES_128_GCM_SHA256", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], ssl_supported_protocols=>["TLSv1.2", "TLSv1.3"], client_inactivity_timeout=>60, executor_threads=>4, event_loop_threads=>0, add_hostname=>false, tls_min_version=>1, tls_max_version=>1.3>
  Error: Address already in use
  Exception: Java::JavaNet::BindException
  Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:555)
sun.nio.ch.ServerSocketChannelImpl.netBind(sun/nio/ch/ServerSocketChannelImpl.java:337)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:294)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:141)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:562)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:600)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:579)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:260)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.runTask(io/netty/util/concurrent/AbstractEventExecutor.java:173)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:166)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:470)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:569)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:997)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:840)
[INFO ] 2024-05-02 18:37:33.838 [[main]<beats] Server - Starting server on port: 5044
[ERROR] 2024-05-02 18:37:39.857 [[main]<beats] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Beats port=>5044, id=>"bd4b83cf08f23a0dabb7fd175613f2eed2178dc3259f903844c0418c7d6817ef", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_14b864b2-c224-4c38-b2ef-7b916c612351", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_enabled=>false, ssl_client_authentication=>"none", ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, ssl_cipher_suites=>["TLS_AES_256_GCM_SHA384", "TLS_AES_128_GCM_SHA256", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], ssl_supported_protocols=>["TLSv1.2", "TLSv1.3"], client_inactivity_timeout=>60, executor_threads=>4, event_loop_threads=>0, add_hostname=>false, tls_min_version=>1, tls_max_version=>1.3>
  Error: Address already in use
  Exception: Java::JavaNet::BindException
  Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:555)
sun.nio.ch.ServerSocketChannelImpl.netBind(sun/nio/ch/ServerSocketChannelImpl.java:337)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:294)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:141)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:562)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:600)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:579)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:260)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.runTask(io/netty/util/concurrent/AbstractEventExecutor.java:173)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:166)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:470)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:569)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:997)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:840)
[INFO ] 2024-05-02 18:37:40.859 [[main]<beats] Server - Starting server on port: 5044
[ERROR] 2024-05-02 18:37:46.877 [[main]<beats] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Beats port=>5044, id=>"bd4b83cf08f23a0dabb7fd175613f2eed2178dc3259f903844c0418c7d6817ef", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_14b864b2-c224-4c38-b2ef-7b916c612351", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, ssl_enabled=>false, ssl_client_authentication=>"none", ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, ssl_cipher_suites=>["TLS_AES_256_GCM_SHA384", "TLS_AES_128_GCM_SHA256", "TLS_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], ssl_supported_protocols=>["TLSv1.2", "TLSv1.3"], client_inactivity_timeout=>60, executor_threads=>4, event_loop_threads=>0, add_hostname=>false, tls_min_version=>1, tls_max_version=>1.3>
  Error: Address already in use
  Exception: Java::JavaNet::BindException
  Stack: sun.nio.ch.Net.bind0(Native Method)
15

Hello, please do not share logs without using the preformatted text option, it makes everything way hard to read.

Use the </> button.

Also, check the error messages, it has hints of the issue.

This means that the port 5044 is already being used. Do you have another Logstash instance running?

16

sorry for the inconvenience idk about this </> button.
no only one logstash is running i have executed the below command

`
sudo netstat -tuln | grep 5044

o/p:
tcp6       0      0 :::5044                 :::*                    LISTEN
` ` `
17

To format your code put 3 back ticks before after your code

```
Your code...
```

1 Like
18

The error in Logstash says that it cannot bind to the port because it is already being used, you need to check what is using the port.

19 
sudo netstat -tuln | grep 5044

o/p:

tcp6       0      0 :::5044                 :::*                    LISTEN

according to this command only one process is running right with that port

20

Which process? Check the PID and see if it is another logstash instance.