# Problem with multiple pipelines

**URL:** <https://discuss.elastic.co/t/problem-with-multiple-pipelines/177035>\
**Category:** Logstash\
**Created:** [April 16, 2019, 8:57am UTC](https://discuss.elastic.co/t/problem-with-multiple-pipelines/177035 "2019-04-16T08:57:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vikash\_Singh1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikash_singh1/32/42119_2.png) [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\
**Post date:** [April 16, 2019, 8:57am UTC](https://discuss.elastic.co/t/problem-with-multiple-pipelines/177035/1 "2019-04-16T08:57:09Z")

</div>

I am trying to import logs from different location from my local machine using logstash simultaneously, I have configured the pipeline.yml as

- pipeline.id: apachelogs  
path.config: "/etc/logstash/conf.d/first-pipeline.conf"

# pipeline.workers: 3

- pipeline.id: squidlogs  
path.config: "/etc/logstash/conf.d/squid\_log.conf"

# queue.type: persisted

- pipeline.id: firewalllogs  
path.config: "/etc/logstash/conf.d/firewall.conf"

# queue.type: persisted

- pipeline.id: dhcpd  
path.config: "/etc/logstash/conf.d/dhcplogs.conf"

# queue.type: persisted

and I am trying to run it as  
/usr/share/logstash/bin/logstash -r /etc/logstash/logstash.yml --config.reload.automatic

Now sometimes it is creating indices but there are only 2 indices created at a moment simulatneously and one of them doesn't have proper data.  
I've also tried using filebeat but in filebeat it inject all the logs under one index  
now the problems are

1. How can I import logs from those location simultaneously?
2. They have to be dynamic so that any time any new logs come it update them automatically

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 14, 2019, 8:57am UTC](https://discuss.elastic.co/t/problem-with-multiple-pipelines/177035/2 "2019-05-14T08:57:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
