# Logstash Setting up multiple config files in one pipeline

**URL:** https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643
**Category:** Logstash
**Created:** [July 16, 2019, 5:28am UTC](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643 "2019-07-16T05:28:55Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![pgw5540](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@pgw5540](https://discuss.elastic.co/u/pgw5540)
#### Post date: [July 16, 2019, 5:28am UTC](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643/1 "2019-07-16T05:28:55Z")

</div>

Hi.

Logstash I want to configure multiple configs on one pipeline.

I want to read dozens of csv files and send data to the same number of different indexes.

So, I created as many config files as there are,  
We want to input specific index data into one pipeline.  
However, when I made several config settings with one pipeline configuration, the data was mixed.

Is there any other way than setting the same number of pipelines?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 16, 2019, 12:37pm UTC](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643/2 "2019-07-16T12:37:00Z")

</div>

If you have multiple configuration files for a single pipeline then they are combined. Events are read from all of the outputs, sent through all of the filters, and every event is written to every output. If you want each configuration file to be self contained then either run it in its own pipeline, or use (for example)

```
add_field => { "documentType" => "typeOne" }

```

on the input and then use

```
if [documentType] == "typeOne" {
    [...]
}

```

in the filter and output sections to make them conditional.

---

<div class="post-metadata">

### Author: ![pgw5540](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@pgw5540](https://discuss.elastic.co/u/pgw5540)
#### Post date: [July 17, 2019, 12:41am UTC](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643/3 "2019-07-17T00:41:00Z")

</div>

thank you

Is there a condition that can make the output different depending on what the input file is?  
Or if there are about 200 Pipelines set up, are there any problems?

I ask for your help.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 17, 2019, 3:51am UTC](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643/4 "2019-07-17T03:51:48Z")

</div>

How different is the processing of the different files? How complex is it?

You may not need a pipeline per filetype, but that depends on the processing.

---

<div class="post-metadata">

### Author: ![pgw5540](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@pgw5540](https://discuss.elastic.co/u/pgw5540)
#### Post date: [July 17, 2019, 4:13am UTC](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643/5 "2019-07-17T04:13:23Z")

</div>

The file extension is the same as csv. However, there are other parts, such as the number of columns and the difference in data types.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 14, 2019, 4:13am UTC](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643/6 "2019-08-14T04:13:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
