# 2 Logstash Conf File, only one works

**URL:** <https://discuss.elastic.co/t/2-logstash-conf-file-only-one-works/101325>\
**Category:** Logstash\
**Created:** [September 21, 2017, 12:03pm UTC](https://discuss.elastic.co/t/2-logstash-conf-file-only-one-works/101325 "2017-09-21T12:03:00Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![feijiangnan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/feijiangnan/32/20893_2.png) [@feijiangnan](https://discuss.elastic.co/u/feijiangnan)\
**Post date:** [September 21, 2017, 12:03pm UTC](https://discuss.elastic.co/t/2-logstash-conf-file-only-one-works/101325/1 "2017-09-21T12:03:01Z")

</div>

Hi there,

I have 2 configuration files to process 2 log files from remote servers with filebeat, each one works by itself, but only one works if both of them are in /etc/logstash/conf.d

Let's call the A and B, A is always works

I got error from B like below:  
[2017-09-20T12:05:48,732][DEBUG][logstash.filters.grok] Event now: {:event=\>2017-09-20T20:00:09.809Z scansrv 2017-09-20 16:00:09,809 [53440178] INFO - ca.toronto.csd.scanagt.listener.ScanAgentMessageCountListener - message added javax.mail.event.MessageCountEvent[source=INBOX] destination /data7/user3 }  
[2017-09-20T12:05:48,733][DEBUG][logstash.util.decorators] filters/LogStash::Filters::Mutate: adding value to field {"field"=\>"read\_timestamp", "value"=\>["%{@timestamp}"]}  
[2017-09-20T12:05:48,733][DEBUG][logstash.filters.geoip] IP was not found in the database {:event=\>2017-09-20T20:00:09.809Z scansrv 2017-09-20 16:00:09,809 [53440178] INFO - ca.toronto.csd.scanagt.listener.ScanAgentMessageCountListener - message added javax.mail.event.MessageCountEvent[source=INBOX] destination /data7/user3 }

Configuration Files:

A:

> input {  
> beats {  
> port =\> "6044"  
> #client\_inactivity\_timeout =\> 6000  
> }  
> }
> 
> filter {  
> grok {  
> match =\> { "message" =\> ["%{IPORHOST:[apache2][access][remote\_ip]} - %{DATA:[apache2][access][user\_name]} [%{HTTPDATE:[apache2][access][time]}] "%{WORD:[apache2][access][method]} %{DATA:[apache2][access][url]} HTTP/%{NUMBER:[apache2][access][http\_version]}" %{NUMBER:[apache2][access][response\_code]} %{NUMBER:[apache2][access][body\_sent][bytes]}( "%{DATA:[apache2][access][referrer]}")?( "%{DATA:[apache2][access][agent]}")?",  
> "%{IPORHOST:[apache2][access][remote\_ip]} - %{DATA:[apache2][access][user\_name]} \[%{HTTPDATE:[apache2][access][time]}\] "-" %{NUMBER:[apache2][access][response\_code]} -" ] }  
> remove\_field =\> "message"  
> }  
> mutate {  
> add\_field =\> { "read\_timestamp" =\> "%{@timestamp}" }  
> }  
> date {  
> match =\> ["[apache2][access][time]", "dd/MMM/YYYY:H:m:s Z" ]  
> remove\_field =\> "[apache2][access][time]"  
> }  
> useragent {  
> source =\> "[apache2][access][agent]"  
> target =\> "[apache2][access][user\_agent]"  
> remove\_field =\> "[apache2][access][agent]"  
> }  
> geoip {  
> source =\> "[apache2][access][remote\_ip]"  
> target =\> "[apache2][access][geoip]"  
> }  
> if "\_grokparsefailure" in [tags] or [apache2][access][user\_name] == "-" {  
> drop { }  
> }  
> }
> 
> output {  
> elasticsearch {  
> hosts =\> ["localhost:9200"]  
> manage\_template =\> false  
> index =\> "svn-apache-%{+YYYY.MM.dd}"  
> document\_type =\> "%{[@metadata][type]}"  
> }  
> stdout { codec =\> rubydebug }  
> }

B:

> # Scan Server MailBot Log
> 
> input {  
> beats {  
> port =\> "5045"  
> }  
> }
> 
> filter {  
> if [fields][document\_type] == "scanserver-mailbot-log" {  
> grok {  
> patterns\_dir =\> "./patterns"  
> break\_on\_match =\> false  
> match =\> { "message" =\> ["%{TIMESTAMP\_ISO8601:timestamp} [%{NUMBER:unknowncode}] %{LOGLEVEL:loglevel} - %{JAVACLASS:classname} - (.\*) destination %{UNIXPATH:userhome}/%{USERNAME:username}" ] }  
> match =\> { "message" =\> ["%{TIMESTAMP\_ISO8601:timestamp} [%{NUMBER:unknowncode}] %{LOGLEVEL:loglevel} - %{JAVACLASS:classname} - Catch exception %{JAVACLASS:exception}[:] %{GREEDYDATA:excepmsg}" ]}  
> remove\_field =\> "message"  
> }
> 
> date {  
> match =\> ["timestamp" , "YYYY-MM-dd HH:mm:ss,SSS"]  
> remove\_field =\> "timestamp"  
> }  
> }  
> }
> 
> output {  
> elasticsearch {  
> hosts =\> ["localhost:9200"]  
> manage\_template =\> false  
> index =\> "%{[fields][document\_type]}-%{+YYYY.MM.dd}"  
> }  
> stdout { codec =\> rubydebug }  
> }

Thanks,  
Fei

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [September 21, 2017, 12:30pm UTC](https://discuss.elastic.co/t/2-logstash-conf-file-only-one-works/101325/2 "2017-09-21T12:30:31Z")

</div>

Up until 6.0 is released with the multiple pipelines feature, Logstash will merge all of your configuration files in /etc/logstash/conf.d into a single, virtual configuration file. That means any filter and output blocks not bounded by conditionals will process events from all inputs. I see only one of your filter blocks bounded by conditionals, and neither of your output blocks are bounded.

---

<div class="post-metadata">

**Author:** ![feijiangnan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/feijiangnan/32/20893_2.png) [@feijiangnan](https://discuss.elastic.co/u/feijiangnan)\
**Post date:** [September 21, 2017, 12:56pm UTC](https://discuss.elastic.co/t/2-logstash-conf-file-only-one-works/101325/3 "2017-09-21T12:56:26Z")

</div>

Thanks for your prompt response, theuntergeek

Added condition, and it works.

Thank you again.

Fei

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 19, 2017, 12:56pm UTC](https://discuss.elastic.co/t/2-logstash-conf-file-only-one-works/101325/4 "2017-10-19T12:56:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.