Still in the initial steps of learning Logstash.
I have written the following pipeline in a configuration file:
input {
stdin {
codec => json
}
http {
}
}
output { stdout{ codec => rubydebug } }
When I run logstash with this pipeline
bin/logstash -f /home/pgangadh/pipeline.conf --path.data PATH
I get the following error:
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2020-10-22 09:02:16.441 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2020-10-22 09:02:16.458 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.8.0"}
[INFO ] 2020-10-22 09:02:25.751 [Converge PipelineAction::Create] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2020-10-22 09:02:25.848 [[main]-pipeline-manager] stdin - Automatically switching from json to json_lines codec {:plugin=>"stdin"}
[INFO ] 2020-10-22 09:02:26.289 [Converge PipelineAction::Create] pipeline - Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x7795ecd5 run>"}
[INFO ] 2020-10-22 09:02:26.300 [[main]<http] http - Starting http input listener {:address=>"0.0.0.0:8080", :ssl=>"false"}
The stdin plugin is now waiting for input:
[INFO ] 2020-10-22 09:02:26.405 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[ERROR] 2020-10-22 09:02:26.597 [[main]<http] pipeline - A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Http id=>"68a7302cd2a7040da73075d3ceebedb116eb8673ca09954ae0efe1b9128f3829", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_590bf824-84c8-405e-b3de-976124d833ff", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", port=>8080, ssl=>false, ssl_verify_mode=>"none", ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], additional_codecs=>{"application/json"=>"json"}, response_headers=>{"Content-Type"=>"text/plain"}, remote_host_target_field=>"host", request_headers_target_field=>"headers", threads=>2, max_pending_requests=>200, max_content_length=>104857600, response_code=>200, verify_mode=>"none">
Error: Address already in use
Exception: Java::JavaNet::BindException
Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:433)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:425)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:223)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:130)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:558)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1358)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:501)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:486)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:1019)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:254)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:366)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:163)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:404)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:462)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:897)
java.lang.Thread.run(java/lang/Thread.java:748)
[INFO ] 2020-10-22 09:02:26.802 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2020-10-22 09:02:27.616 [[main]<http] http - Starting http input listener {:address=>"0.0.0.0:8080", :ssl=>"false"}
[ERROR] 2020-10-22 09:02:27.623 [[main]<http] pipeline - A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Http id=>"68a7302cd2a7040da73075d3ceebedb116eb8673ca09954ae0efe1b9128f3829", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_590bf824-84c8-405e-b3de-976124d833ff", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", port=>8080, ssl=>false, ssl_verify_mode=>"none", ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], additional_codecs=>{"application/json"=>"json"}, response_headers=>{"Content-Type"=>"text/plain"}, remote_host_target_field=>"host", request_headers_target_field=>"headers", threads=>2, max_pending_requests=>200, max_content_length=>104857600, response_code=>200, verify_mode=>"none">
I can make out some errors. It starts Logstash, pipeline is started successfully, starts http plugin listener, and then fails.
If I remove the http plugin from the pipeline, there wont be any error. Please help