# Multiple logstash configs with different filters running at the same time

**URL:** https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749
**Category:** Logstash
**Created:** [June 25, 2020, 7:14pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749 "2020-06-25T19:14:17Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Archie\_Crawford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/archie_crawford/32/62732_2.png) [@Archie\_Crawford](https://discuss.elastic.co/u/Archie_Crawford)
#### Post date: [June 25, 2020, 7:14pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749/1 "2020-06-25T19:14:17Z")

</div>

Hello, I have been having this issue when I try to send server logs to logstash..

Java stack trace server logs sent to Logstash with their own input.conf filter.conf and output.conf

and

Cisco ASA firewall logs going to the same Logstash with their own input.conf filter.conf and output.conf

When I do both of these at the same time in the same logstash the data gets mixed up some times and Ill see firewall logs in the java stack trace index.. or vice versa

Will adding more pipeline workers fix this ?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 25, 2020, 8:39pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749/2 "2020-06-25T20:39:00Z")

</div>

Running the two configurations in different pipelines might. it depends on how you have configured logstash.

---

<div class="post-metadata">

### Author: ![Archie\_Crawford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/archie_crawford/32/62732_2.png) [@Archie\_Crawford](https://discuss.elastic.co/u/Archie_Crawford)
#### Post date: [June 26, 2020, 1:53pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749/3 "2020-06-26T13:53:53Z")

</div>

So how do you separate the pipelines? because right now I have two .. but the input from one is going to both indexes . and I have 4 pipeline workers

---

<div class="post-metadata">

### Author: ![Archie\_Crawford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/archie_crawford/32/62732_2.png) [@Archie\_Crawford](https://discuss.elastic.co/u/Archie_Crawford)
#### Post date: [June 26, 2020, 1:54pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749/4 "2020-06-26T13:54:51Z")

</div>

```auto
  beats {
    port => 15044
 }
}

filter {
}

output {
  elasticsearch {
  hosts => ["xxxxx:9500"]
  manage_template => false
  index => "devtest-%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
 }
}

```

---

<div class="post-metadata">

### Author: ![Archie\_Crawford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/archie_crawford/32/62732_2.png) [@Archie\_Crawford](https://discuss.elastic.co/u/Archie_Crawford)
#### Post date: [June 26, 2020, 1:55pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749/5 "2020-06-26T13:55:37Z")

</div>

```auto

input {
  beats {
    port => 5045
  }
}
filter {
}

output {
  elasticsearch {
  hosts => ["xxxxxx:9500"]
  manage_template => false
  index => "sdttest-%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
 }
}

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 26, 2020, 2:47pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749/6 "2020-06-26T14:47:50Z")

</div>

Does [this](https://discuss.elastic.co/t/conf-files-getting-mixed-when-uploading/195803/2) help?

---

<div class="post-metadata">

### Author: ![Archie\_Crawford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/archie_crawford/32/62732_2.png) [@Archie\_Crawford](https://discuss.elastic.co/u/Archie_Crawford)
#### Post date: [June 26, 2020, 3:14pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749/7 "2020-06-26T15:14:21Z")

</div>

Yes .. basically this has to be in the pipeline.yml..

```auto
- pipeline.id: main
  path.config: "/etc/logstash/conf.d/15044-pipeline.conf"
  pipeline.workers: 2
- pipeline.id: secondary
  path.config: "/etc/logstash/conf.d/5045-test-pipeline.conf"
  queue.type: persisted

```

It seems to be working but Ill be doing more test on it over this weekend .. thank you

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 24, 2020, 3:14pm UTC](https://discuss.elastic.co/t/multiple-logstash-configs-with-different-filters-running-at-the-same-time/238749/8 "2020-07-24T15:14:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
