# Issue with Multiple .conf files

**URL:** <https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760>\
**Category:** Logstash\
**Created:** [September 7, 2020, 10:26am UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760 "2020-09-07T10:26:06Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![anirudh.venkata](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@anirudh.venkata](https://discuss.elastic.co/u/anirudh.venkata)\
**Post date:** [September 7, 2020, 10:26am UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760/1 "2020-09-07T10:26:06Z")

</div>

Hi All,  
My ELK is setup in a single Ubuntu machine.. I am using Logstash as a service to collect different logs and send them to Elasticsearch..Initially ,i have created an auth.conf in logstash conf.d folder to parse Linux auth logs .After restarting Logstash, I could see the index (Auth logs index) getting created in Kibana. Then i was trying for my application logs..so i stopped logstash, removed auth.conf file and added application.conf file in conf.d folder...After restarting logstash , i could see the new index (application logs index) getting created in Kibana..Later i found that Logstash processes all the .conf files in the conf.d folder..So i stopped logstash,deleted the previous 2 indices and removed the .sincedb files so that it starts from the beginning of file..I placed both the auth.conf and application.conf files in conf.d folder and restarted logstash, but i don't see any index getting created in Kibana..

Please assist me in resolving the issue, also let me know if you need more information.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 7, 2020, 10:03pm UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760/2 "2020-09-07T22:03:54Z")

</div>

Welcome to our community! 😃  
Can you please edit your post and remove the formatting, it makes it very hard to read and help you.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 8, 2020, 6:32am UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760/4 "2020-09-08T06:32:08Z")

</div>

Thanks for that, can you share your configs?

---

<div class="post-metadata">

**Author:** ![anirudh.venkata](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@anirudh.venkata](https://discuss.elastic.co/u/anirudh.venkata)\
**Post date:** [September 8, 2020, 7:09am UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760/5 "2020-09-08T07:09:49Z")

</div>

This is my auth.conf

input {  
file {  
path =\> "/opt/logstash/auth.log"  
type =\> "syslog"  
start\_position =\> "beginning"  
}  
}

filter{  
grok{  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname} sshd(?:[%{POSINT:system.auth.pid}])?: %{GREEDYDATA:system.auth.ssh.event} %{GREEDYDATA:system.auth.ssh.method} from %{IPORHOST:system.auth.ip} port %{NUMBER:system.auth.port}:%{GREEDYDATA:typo}" }  
}  
mutate{  
convert =\> { "bytes" =\> "integer" }  
}  
geoip {  
source =\> "system.auth.ip"  
}  
if "\_grokparsefailure" in [tags] {  
drop { }  
}  
}

output {  
elasticsearch {  
hosts =\> "localhost:9200"  
index =\> "logstash-auth"  
}  
}

This is my application.conf

input {  
file {  
path =\> "/opt/logstash/services.log4j2.log"  
start\_position =\> "beginning"  
}  
}

filter{  
grok{  
match =\> { "message" =\> "(?[(._?)]) %{TIMESTAMP\_ISO8601:TIME} (?[(._?)]) %{WORD:Methodname} %{GREEDYDATA:Messagebody}"}  
}  
}

output {  
elasticsearch {  
hosts =\> "localhost:9200"  
index =\> "application"  
}  
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 9, 2020, 12:14am UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760/6 "2020-09-09T00:14:01Z")

</div>

Please format your code/logs/config using the `</>` button, or markdown style back ticks. It helps to make things easy to read which helps us help you 🙂

The thing to take into account is that Logstash will merge both config files into one when it starts up. You will want to use conditionals to match each input with its own filter and output.

---

<div class="post-metadata">

**Author:** ![anirudh.venkata](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@anirudh.venkata](https://discuss.elastic.co/u/anirudh.venkata)\
**Post date:** [September 9, 2020, 11:24am UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760/7 "2020-09-09T11:24:43Z")

</div>

Thanks Mark for the followup, i tried using if conditional in both the .conf files as below, and used the /usr/share/logstash/bin/logstash -f /path/to/syslog.conf -f /path/to/application.conf ,but i see only the application.conf index getting created.

This is my application.conf

```
input {
  file {
    path => "/opt/logstash/services.log4j2.log"
    tags => ["applicationdata"]
    start_position => "beginning"
  }
}

filter{
  if "applicationdata" in [tags]{
     grok{
       match => { "message" => "(?<INFO>\[(.*?)\]) %{TIMESTAMP_ISO8601:TIME} (?<Classname>\[(.*?)\]) %{WORD:Methodname} %{GREEDYDATA:Messagebody}" }
     }
  }
}

output {
  if "applicationdata" in [tags]{
     elasticsearch {
       hosts => "localhost:9200"
       index => "application"
     }
  }
}

```

This is my syslog.conf

```
input {
 file {
   path => "/opt/logstash/syslog.log"
   tags => ["syslogdata"]
   start_position => "beginning"
 }
}

filter{
  if "syslogdata" in [tags]{
     grok{
       match => { "message" => "%{SYSLOGTIMESTAMP:systemtimestamp} %{SYSLOGHOST:systemhostname} %{WORD:Methodname}(?:\[%{POSINT:systempid}\])?: %{GREEDYDATA:servicename}: %{GREEDYDATA:Message}" }
       match => { "message" => "%{SYSLOGTIMESTAMP:systemtimestamp} %{SYSLOGHOST:systemhostname} %{WORD:Methodname}(?:\[%{POSINT:systempid}\])?: %{GREEDYDATA:Message}" }
     }
     if "_grokparsefailure" in [tags] {
          drop { }
     }
  }
}

output {
  if "syslogdata" in [tags]{
     elasticsearch {
       hosts => "localhost:9200"
       index => "slog${DATE}${TIME}"
     }
  }
}
```

---

<div class="post-metadata">

**Author:** ![anirudh.venkata](https://avatars.discourse-cdn.com/v4/letter/a/48db29/32.png) [@anirudh.venkata](https://discuss.elastic.co/u/anirudh.venkata)\
**Post date:** [September 17, 2020, 11:31am UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760/8 "2020-09-17T11:31:33Z")

</div>

Found an alternate to this though...created pipeline for each conf file in the pipelines.yml and after restarting logstash service ,was able to see all new indices getting created in kibana.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 15, 2020, 11:31am UTC](https://discuss.elastic.co/t/issue-with-multiple-conf-files/247760/9 "2020-10-15T11:31:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.