Multiple pipelines and multiple filebeats

Can you please help me below.

  1. I have multiple pipelines. each pipleline configuration is receives data from different application servers where we installed filebeat.

  2. When i have single pipeline config running(on logstash server) and single filebeat running(on app server), we are able to receive the logs to logstash server.

  3. I have new pipelined added which should receive log data from another application server. Here another filebeat instance is running.

When two separate filebeat instances are sending data to two different logstash pipelines(both are having beats input, how logstash know which pipeline should receive data from which filebeat server.

Here is the error i am getting

[2018-07-27T15:38:07,570][INFO ][] Starting server on port: 5044
[2018-07-27T15:38:13,787][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id: api_filebeat
Plugin: <LogStash::Inputs::Beats port=>5044, client_inactivity_timeout=>120, id=>"9aa966e55046c21f0e07eb96962fd976dc46e3f80a9205e44799e10da8703c99", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_e8908910-e10f-40bd-a5ab-fbb7d3a00f5b", enable_metric=>true, charset=>"UTF-8">, host=>"", ssl=>false, ssl_verify_mode=>"none", include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], executor_threads=>4>
Error: Address already in use
Exception: Java::JavaNet::BindException
Stack: Method)$AbstractUnsafe.bind(io/netty/channel/$HeadContext.bind(io/netty/channel/

It says address already in use but it is not in use. Single logstash service with multiple pipelines. Each pipeline receives data from multiple app servers(each has its own filebeat running)

Could you please help what to do when we have this situation.

You can't have multiple beats inputs (in the same pipeline or in different pipelines) that listen on the same TCP port. If you want to separate processing of events from different applications in different pipelines you need to ship the events to different ports on the Logstash host.


Thank you very much Magnus. I changed TCP port from default. It worked .

Thank you.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.