# Config files for mutliple pipelines

**URL:** <https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976>\
**Category:** Logstash\
**Created:** [January 18, 2018, 6:39am UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976 "2018-01-18T06:39:38Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![shubham\_kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_kumar1/32/22084_2.png) [@shubham\_kumar1](https://discuss.elastic.co/u/shubham_kumar1)\
**Post date:** [January 18, 2018, 6:39am UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976/1 "2018-01-18T06:39:38Z")

</div>

Hi,  
I am trying to configure two separate pipelines in logstash-6.1.1 through pipeline.yml. For each configuration file i am specifying same input and output.

1st\_pipeline.conf:-  
input{  
beats{  
port =\> 5044  
}  
}

filter{  
....................  
}

output{  
stdout{ codec =\> rubydebug}  
}

2nd\_pipeline.conf:-  
input{  
beats{  
port =\> 5044  
}  
}

filter{  
.................................  
}

output{  
stdout{ codec =\> rubydebug }  
}

Now as i am trying to run both of these pipeline together its throwing error

Plugin: \<LogStash::Inputs::Beats port=\>5044, id=\>"289fbe7dcd840f0b3bf8e6a2cc6ef3f98776d60fcb8cc214b41d85f129b283a3", enable\_metric=\>true, codec=\>\<LogStash::Codecs::Plain id=\>"plain\_2796f793-4d5f-4c9c-a464-86c091b70277", enable\_metric=\>true, charset=\>"UTF-8"\>, host=\>"0.0.0.0", ssl=\>false, ssl\_verify\_mode=\>"none", include\_codec\_tag=\>true, ssl\_handshake\_timeout=\>10000, tls\_min\_version=\>1, tls\_max\_version=\>1.2, cipher\_suites=\>["TLS\_ECDHE\_ECDSA\_WITH\_AES\_256\_GCM\_SHA384", "TLS\_ECDHE\_RSA\_WITH\_AES\_256\_GCM\_SHA384", "TLS\_ECDHE\_ECDSA\_WITH\_AES\_128\_GCM\_SHA256", "TLS\_ECDHE\_RSA\_WITH\_AES\_128\_GCM\_SHA256", "TLS\_ECDHE\_ECDSA\_WITH\_AES\_256\_CBC\_SHA384", "TLS\_ECDHE\_RSA\_WITH\_AES\_256\_CBC\_SHA384", "TLS\_ECDHE\_RSA\_WITH\_AES\_128\_CBC\_SHA256", "TLS\_ECDHE\_ECDSA\_WITH\_AES\_128\_CBC\_SHA256"], client\_inactivity\_timeout=\>60, executor\_threads=\>16\>  
Error: Address already in use

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 18, 2018, 6:43am UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976/2 "2018-01-18T06:43:59Z")

</div>

You can not have two different inputs listening on the same port within a Logstash instance, even if they belong to different pipelines. You will therefore need to change the port for one of them.

---

<div class="post-metadata">

**Author:** ![shubham\_kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_kumar1/32/22084_2.png) [@shubham\_kumar1](https://discuss.elastic.co/u/shubham_kumar1)\
**Post date:** [January 18, 2018, 6:46am UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976/3 "2018-01-18T06:46:52Z")

</div>

i changed the the port it works properly.  
thanks

---

<div class="post-metadata">

**Author:** ![Milad\_Hamid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milad_hamid/32/24327_2.png) [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)\
**Post date:** [January 18, 2018, 12:46pm UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976/4 "2018-01-18T12:46:31Z")

</div>

Hi @Christian_Dahlqvist.

If we changed the port for one of pipelines what about the filebeat.yml do we need to change anything in the logstash output. for example my filebeat.yml has output.logstash as follow:  
output.logstash:  
# The Logstash hosts  
hosts: ["logstashdomain:5044"]  
and my filebeat shipping three different log events (apache error, apache acces, mysql error)

and on the logstash server I have three pipelines  
and each of this conf file has beats input with port 5044. So if i give unique port to each of them do i need to edit my output.logstash in filebeat.yml for example:  
output.logstash:  
# The Logstash hosts  
hosts: ["logstashdomain:5044", "logstashdomain:5045", "logstashdomain:5046"]

Is it correct?

---

<div class="post-metadata">

**Author:** ![Milad\_Hamid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milad_hamid/32/24327_2.png) [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)\
**Post date:** [January 18, 2018, 2:11pm UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976/5 "2018-01-18T14:11:50Z")

</div>

Hi @shubham_kumar1 can you please help with this since you had experience with this issue.

If we changed the port for one of pipelines what about the filebeat.yml do we need to change anything in the logstash output. for example my filebeat.yml has output.logstash as follow:  
output.logstash:  
# The Logstash hosts  
hosts: ["logstashdomain:5044"]  
and my filebeat shipping three different log events (apache error, apache acces, mysql error)

and on the logstash server I have three pipelines  
and each of this conf file has beats input with port 5044. So if i give unique port to each of them do i need to edit my output.logstash in filebeat.yml for example:  
output.logstash:  
# The Logstash hosts  
hosts: ["logstashdomain:5044", "logstashdomain:5045", "logstashdomain:5046"]

Is it correct?

---

<div class="post-metadata">

**Author:** ![shubham\_kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_kumar1/32/22084_2.png) [@shubham\_kumar1](https://discuss.elastic.co/u/shubham_kumar1)\
**Post date:** [February 2, 2018, 9:25am UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976/6 "2018-02-02T09:25:58Z")

</div>

Yes you need to update the port in filebeat also. logstash will only listen to the port mentioned in input.

---

<div class="post-metadata">

**Author:** ![Milad\_Hamid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milad_hamid/32/24327_2.png) [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)\
**Post date:** [February 2, 2018, 9:46am UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976/7 "2018-02-02T09:46:39Z")

</div>

thank you @shubham_kumar1 for your help. Actually we decided to use two filebeat instances.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 2, 2018, 9:46am UTC](https://discuss.elastic.co/t/config-files-for-mutliple-pipelines/115976/8 "2018-03-02T09:46:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
