# One Logstash file with multiple logs entries/pipelines

**URL:** https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991
**Category:** Logstash
**Created:** [November 24, 2017, 8:51am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991 "2017-11-24T08:51:38Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Milad\_Hamid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milad_hamid/32/24327_2.png) [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)
#### Post date: [November 24, 2017, 8:51am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/1 "2017-11-24T08:51:39Z")

</div>

Hi,

I have two different logs source ( Syslogs;logstash-syslog.conf, Apache logs;logstash-apache.conf), and i want to get the syslogs and apache logs into logstash and then ship them to elasticsearch with one logstash instance. So how i can implement this request instead of each time running this command to ship my logs to logstash:

bin/logstash -f path/config/logstash-apache.conf OR bin/logstash -f path/config/logstash-syslog.conf

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 24, 2017, 8:57am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/2 "2017-11-24T08:57:05Z")

</div>

If you are using Logstash 6.0, you can set it up to manage [multiple pipelines](https://www.elastic.co/guide/en/logstash/6.0/multiple-pipelines.html) in parallel based on your two configurations. In earlier versions you could place both files in a directory and point Logstash to this instead of the files. Logstash would then concatenate the files into a single pipeline (be sure you are using [conditionals](https://www.elastic.co/guide/en/logstash/6.0/event-dependent-configuration.html#event-dependent-configuration) to separate the two flows).

---

<div class="post-metadata">

### Author: ![Milad\_Hamid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milad_hamid/32/24327_2.png) [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)
#### Post date: [November 24, 2017, 10:20am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/3 "2017-11-24T10:20:01Z")

</div>

Thanks @Christian_Dahlqvist for your quick reply and help. The version that i use right now in my ELK stack is Logstash 5.6.4.

Since I am new to ELK world, please i want some basics steps how to implement the second approach realetd to old version of logstash to manage two config files. This is a screenshot of my Logstash/Config directory:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/8/58b32852facba1540f18033b40063a797ea0edf5.png)

So what i have to edit and to do to get it run.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 24, 2017, 10:22am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/4 "2017-11-24T10:22:39Z")

</div>

Put the config files in a separate directory and pass this to Logstash at startup. Make sure you have tagged the data in the filters and applied conditionals throughout before you do that though.

---

<div class="post-metadata">

### Author: ![Milad\_Hamid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milad_hamid/32/24327_2.png) [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)
#### Post date: [November 24, 2017, 10:40am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/5 "2017-11-24T10:40:15Z")

</div>

Ok, so i have to put each config file in separate directory, then in logstash yaml file i have to add the paths of directories! which section in logstash settings file i have to edit to point to the directories? is it Pipeline Configuration Settings in logstash.yml file?

and regarding tags in filter part you meant this:  
filter {  
if [type] == "type-of-message-from-redis" {  
...  
}  
if [type] == "type-of-message-from-file" {  
...  
}  
}

output {  
if [type] == "type-of-message-from-redis" {  
elasticsearch {  
host =\> "localhost"  
}  
}  
if [type] == "type-of-message-from-file" {  
elasticsearch {  
host =\> "localhost"  
index =\> "ssllogs-%{+YYYY.MM.dd}"  
}  
}  
}

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 24, 2017, 10:54am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/6 "2017-11-24T10:54:47Z")

</div>

You put both files in the same directory (no other files) and then give that directory instead of a file name when you start Logstash.

---

<div class="post-metadata">

### Author: ![Milad\_Hamid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milad_hamid/32/24327_2.png) [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)
#### Post date: [November 24, 2017, 10:57am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/7 "2017-11-24T10:57:35Z")

</div>

Ok, thanks i will test and post the result in case of success

---

<div class="post-metadata">

### Author: ![Milad\_Hamid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milad_hamid/32/24327_2.png) [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)
#### Post date: [November 24, 2017, 11:43am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/8 "2017-11-24T11:43:52Z")

</div>

Yes thanks it is working in great way i have both logs now in logstash, and i can see data in Kibana.

So just to summarize because i know there are a lot people like me new to ELK, so I created a directory with name pipelines and i moved my logstash-syslog.conf and logstash-apache.conf files into this new directory, then i used following command to run logstash and ship my logs:  
bin/logstash -f path/logstash-5.6.4/pipelines

Thank a lot @Christian_Dahlqvist

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 22, 2017, 11:43am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991/9 "2017-12-22T11:43:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
