# Multi pipelines

**URL:** <https://discuss.elastic.co/t/multi-pipelines/243676>\
**Category:** Logstash\
**Created:** [August 4, 2020, 9:17am UTC](https://discuss.elastic.co/t/multi-pipelines/243676 "2020-08-04T09:17:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![peretz\_shlomi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/peretz_shlomi/32/54701_2.png) [@peretz\_shlomi](https://discuss.elastic.co/u/peretz_shlomi)\
**Post date:** [August 4, 2020, 9:17am UTC](https://discuss.elastic.co/t/multi-pipelines/243676/1 "2020-08-04T09:17:59Z")

</div>

hi  
i have two pipeline that collect csv file from different folder  
each folder collect the data to a different index  
The file name is the same (Inside the file the information is a bit different)  
i have problem that the logstash collect only from one folder

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [August 4, 2020, 9:23am UTC](https://discuss.elastic.co/t/multi-pipelines/243676/2 "2020-08-04T09:23:26Z")

</div>

Please post the content of your pipelines.yml, the .conf files and Logstash logs. An abstract description of the problem is not enough to solve this.

---

<div class="post-metadata">

**Author:** ![peretz\_shlomi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/peretz_shlomi/32/54701_2.png) [@peretz\_shlomi](https://discuss.elastic.co/u/peretz_shlomi)\
**Post date:** [August 4, 2020, 12:34pm UTC](https://discuss.elastic.co/t/multi-pipelines/243676/3 "2020-08-04T12:34:23Z")

</div>

hi

pipeline.yml

- pipeline.id: logstash1  
path.config: "/kibana/logstash-7.6.1/logstash-7.6.1/bin/p1/logstash1.conf"  
pipeline.workers: 3  
pipeline.batch.size: 5
- pipeline.id: logstash2  
path.config: "/kibana/logstash-7.6.1/logstash-7.6.1/bin/p2/logstash2.conf"  
pipeline.workers: 3  
queue.type: persisted  
pipeline.batch.size: 5

conf

input {  
file {  
path =\> "c:/input/\*.csv"  
start\_position =\> "beginning"

}

}

filter {  
csv {  
separator =\> ","  
columns =\> ["time","file type","Product name","Side","board result","Machine NAME","Barcode","Barcode slave","Component Qty","Real Component Qty","NG Amount","NG Qty","Operator","Working Order","module number","location","Matiral NO","NG name","NG result"]  
}  
mutate {convert =\> ["Component Qty" , "integer"] }  
mutate {convert =\> ["Real Component Qty" , "integer"] }  
mutate {convert =\> ["NG Amount" , "integer"] }  
mutate {convert =\> ["NG Qty" , "integer"] }  
date {  
match =\> ["time", "yyyyMMdd HHmmss"]  
timezone =\> "UTC"  
}  
}

output {

elasticsearch {  
hosts =\> ["127.0.0.1:9200"]  
user =\> "elastic"  
password =\> "sLf7r0eAsdKD4XPxoWpO"  
index =\> "aoi\_pemtron"

}  
stdout {}  
}

i dont have error at the cmd window , It just does not import the file

---

<div class="post-metadata">

**Author:** ![peretz\_shlomi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/peretz_shlomi/32/54701_2.png) [@peretz\_shlomi](https://discuss.elastic.co/u/peretz_shlomi)\
**Post date:** [August 7, 2020, 5:31am UTC](https://discuss.elastic.co/t/multi-pipelines/243676/4 "2020-08-07T05:31:00Z")

</div>

someone ? Plz

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [August 7, 2020, 1:24pm UTC](https://discuss.elastic.co/t/multi-pipelines/243676/5 "2020-08-07T13:24:39Z")

</div>

Hi @peretz_shlomi,

Do you have the same `input` config on both pipelines?

> [@peretz\_shlomi](#):
>
> path =\> "c:/input/\*.csv"

Especially curious if you have the same `path`. You did say "collect csv file from different folder" so I guess not..?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 4, 2020, 1:24pm UTC](https://discuss.elastic.co/t/multi-pipelines/243676/6 "2020-09-04T13:24:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.