Hello Logstash community members,
I am relatively new to logstash and want to read data sent by TCP output plugin in my java code. My conf file is shown below.
input
{
tcp
{
port => 5518
type => "syslogType"
}
}
filter{
grok{
// some grok is here.
}
}
output{
tcp
{
host => "localhost"
port => 5519
mode => "server"
codec => "json_lines"
}
file {
path => "/tmp/preview.txt"
codec => "json"
}
}
This works fine but I also have a requirement to change the grok written in the conf file without logstash restart. To achieve this I edit conf file on the fly using vi editor. I have config.reload.automatic: true and have 2 pipelines defined in pipelines.yml in logstash version 6.5.3 OSS on my Ubuntu machine. When Logstash tries to reload the configuration, I get below error.
Jan 10 17:30:57 my-pc logstash[30898]: [2019-01-10T17:30:57,413][INFO ][logstash.pipelineaction.reload] Reloading pipeline {"pipeline.id"=>:preview_pipeline}
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,493][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{}}
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,501][ERROR][org.logstash.execution.ShutdownWatcherExt] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,802][INFO ][logstash.pipeline ] Pipeline has terminated {:pipeline_id=>"preview_pipeline", :thread=>"#<Thread:0x73a79803 run>"}
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,925][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"preview_pipeline", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,932][INFO ][logstash.outputs.tcp ] Starting tcp output listener {:address=>"localhost:5519"}
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,936][ERROR][logstash.outputs.tcp ] Could not start TCP server: Address in use {:host=>"localhost", :port=>5519}
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,944][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"preview_pipeline", :plugin=>"#<LogStash::OutputDelegator:0x37af5e47>", :error=>"Address already in use - bind - Address already in use", :thread=>"#<Thread:0x62072031 run>"}
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,949][ERROR][logstash.pipeline ] Pipeline aborted due to error {:pipeline_id=>"preview_pipeline", :exception=>#<Errno::EADDRINUSE: Address already in use - bind - Address already in use>, :backtrace=>["org/jruby/ext/socket/RubyTCPServer.java:127:in `initialize'", "org/jruby/RubyIO.java:875:in `new'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-tcp-5.0.3/lib/logstash/outputs/tcp.rb:115:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:102:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:46:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:594:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:200:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:160:in `block in start'"], :thread=>"#<Thread:0x62072031 run>"}
Jan 10 17:31:02 my-pc logstash[30898]: [2019-01-10T17:31:02,961][ERROR][logstash.agent ] Failed to execute action {:id=>:preview_pipeline, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<preview_pipeline>, action_result: false", :backtrace=>nil}
Can someone please help me identify why on reloading configuration logstash does not release the port 5519 which is my output port ? If I stop the logstash service and start it again then everything works fine. I want same to happen with conf reload. Please help. Thanks.