# Many Conditional Checks in Logstash Conf

**URL:** https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934
**Category:** Logstash
**Created:** [June 3, 2019, 10:27am UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934 "2019-06-03T10:27:19Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![sjabiulla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sjabiulla/32/48429_2.png) [@sjabiulla](https://discuss.elastic.co/u/sjabiulla)
#### Post date: [June 3, 2019, 10:27am UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934/1 "2019-06-03T10:27:19Z")

</div>

Hi,

Will it be a performance impact If I have many conditional statements in Logstash config ?

My idea was to create a single Logstash setup whose input is FileBeat, WinLogBeat and HeartBeat. Based on the beat type and source log I am planning to push to different indexes in ElasticSearch. My have now finished a small POC and I have a bunch of if-else conditions in my conf file. So Wanted to check, will it impact the performance of Logstash ?

---

<div class="post-metadata">

### Author: ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)
#### Post date: [June 3, 2019, 11:44am UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934/2 "2019-06-03T11:44:36Z")

</div>

Why do you want to keep all in one config file instead of running separate ones per beat family product?  
Input is not multi-threaded, only filter processing gets distributed over multiple threads, so that will be the limitation here. If you separate this you will get performance boost.

From my experience I did not see any noticeable performance impact with simple if conditions.  
(My EPS was 3000-4000).

---

<div class="post-metadata">

### Author: ![sjabiulla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sjabiulla/32/48429_2.png) [@sjabiulla](https://discuss.elastic.co/u/sjabiulla)
#### Post date: [June 3, 2019, 11:56am UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934/3 "2019-06-03T11:56:25Z")

</div>

Thanks for the reply.

> [@pastechecker](#):
>
> Why do you want to keep all in one config file instead of running separate ones per beat family product?

From my understanding, If we have multiple config files ( May be one per each beat family ), it just increases readability. But logstash concatenates and runs as a single config . So when logstash runs all these config files, it's going to have same conditional operators irrespective of single huge config file or multiple config files !!

> [@pastechecker](#):
>
> Input is not multi-threaded, only filter processing gets distributed over multiple threads, so that will be the limitation here. If you separate this you will get performance boost.

Could you please elaborate it a bit more. I didn't get this part.

---

<div class="post-metadata">

### Author: ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)
#### Post date: [June 3, 2019, 12:04pm UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934/4 "2019-06-03T12:04:10Z")

</div>

When the pipeline starts Logstash assigns workers to the filter and the output processing stage.  
Number of workers by default assumes the number of the host’s CPU cores. I think that you process input one by one and then spread across multiple threads when you pass the data to the filter section.

Have a look also on the: [https://www.elastic.co/guide/en/logstash/current/logstash-settings-file.html](https://www.elastic.co/guide/en/logstash/current/logstash-settings-file.html)

To determine the performance impact on your machine you can use the stats new API. [https://www.elastic.co/guide/en/logstash/current/node-stats-api.html](https://www.elastic.co/guide/en/logstash/current/node-stats-api.html)

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 3, 2019, 1:18pm UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934/5 "2019-06-03T13:18:40Z")

</div>

> [@sjabiulla](#):
>
> From my understanding, If we have multiple config files ( May be one per each beat family ), it just increases readability. But logstash concatenates and runs as a single config . So when logstash runs all these config files, it's going to have same conditional operators irrespective of single huge config file or multiple config files !!

Not if you use the [multiple pipelines feature](https://www.elastic.co/guide/en/logstash/7.1/multiple-pipelines.html) and [pipeline to pipeline communication](https://www.elastic.co/guide/en/logstash/7.1/pipeline-to-pipeline.html).

> [@sjabiulla](#):
>
> Could you please elaborate it a bit more. I didn't get this part.

Having separate pipelines per input will ensure they run on separate threads and do not block eachother.

---

<div class="post-metadata">

### Author: ![sjabiulla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sjabiulla/32/48429_2.png) [@sjabiulla](https://discuss.elastic.co/u/sjabiulla)
#### Post date: [June 11, 2019, 12:29pm UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934/6 "2019-06-11T12:29:39Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> Not if you use the [multiple pipelines feature](https://www.elastic.co/guide/en/logstash/7.1/multiple-pipelines.html)

Thanks for the info @Christian_Dahlqvist. I'm now using multiple pipelines feature. But If I have same input.beats.port in two config files, I was getting "Address Already Bind" error while starting logstash. When I used different ports, it worked. Is this expected ?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 11, 2019, 3:50pm UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934/7 "2019-06-11T15:50:06Z")

</div>

Yes, you can only have one process listen to a single port, so all inputs need to use different ports. You could also create a separate pipeline that contains the input plugin and the sends data to different pipelines based on fields in the data. Each pipeline does not need to contain all types of plugins.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 9, 2019, 3:50pm UTC](https://discuss.elastic.co/t/many-conditional-checks-in-logstash-conf/183934/8 "2019-07-09T15:50:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
