# Multiple filter files for different host

**URL:** <https://discuss.elastic.co/t/multiple-filter-files-for-different-host/49262>\
**Category:** Logstash\
**Created:** [May 5, 2016, 7:20am UTC](https://discuss.elastic.co/t/multiple-filter-files-for-different-host/49262 "2016-05-05T07:20:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Muhasin\_Cm](https://avatars.discourse-cdn.com/v4/letter/m/a6a055/32.png) [@Muhasin\_Cm](https://discuss.elastic.co/u/Muhasin_Cm)\
**Post date:** [May 5, 2016, 7:20am UTC](https://discuss.elastic.co/t/multiple-filter-files-for-different-host/49262/1 "2016-05-05T07:20:52Z")

</div>

Hi ,

I have installed ELK stack on a linux machine and sending events using filebeat from different files.I have installed filebeat on different machines.

The process is like below

filebeat =\> logstash shipper =\> redis =\> elastic search

I have logstash shipper on centralized server for receiving log events from filebeat. filebeat send data to logstash shipper on the port : 5044 and shipper is running on the port.

I need to set up different configuration files of shipper for the machines sending data from filebeat. for example machine 1 should have machine1 shipper configuration.

Is it possible to use multiple logstash configuration for different machines using one single port or is there an alternative method for it.

Am getting port conflict for it when i have tried it,

I have also tried to set up multiple filter configuration file for different machines with one single input and output,but it does not seems to be working. The logs are not parsing via the filter.

anybody have a solution to this?.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 5, 2016, 7:27am UTC](https://discuss.elastic.co/t/multiple-filter-files-for-different-host/49262/2 "2016-05-05T07:27:38Z")

</div>

Use conditionals based on variables, like the hostname - [https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#conditionals](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#conditionals)

---

<div class="post-metadata">

**Author:** ![Muhasin\_Cm](https://avatars.discourse-cdn.com/v4/letter/m/a6a055/32.png) [@Muhasin\_Cm](https://discuss.elastic.co/u/Muhasin_Cm)\
**Post date:** [May 5, 2016, 7:32am UTC](https://discuss.elastic.co/t/multiple-filter-files-for-different-host/49262/3 "2016-05-05T07:32:35Z")

</div>

Hi Warkolm,

I have used the conditional hostname and created two filter files inside the logstash configuration directory.

but for some reason logstash does not working with two filter files at a time.

# filter configuration for one machine

filter  
{  
if [host] == "datanode2"  
{

if [type] == "log"  
{  
grok  
{  
match =\> ["message", "%{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}: %{WORD:authentication}"]  
add\_tag =\> ["log", "grokked"]  
}  
}

if [type] == "log" and [authentication] not in ["Failed","info"]  
{  
drop{}  
}

if [type] == "iisbeat"  
{  
#ignore log comments  
if [message] =~ "^#" {  
drop {}  
}

grok {  
# check that fields match your IIS log settings  
match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{IPORHOST:hostip} %{WORD:method} %{NOTSPACE:page} %{NOTSPACE:query} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clientip} %{NOTSPACE:useragent} %{NOTSPACE:referrer} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:scstatus} %{NUMBER:timetaken}"]  
}  
}

#Set the Event Timesteamp from the log  
date {  
match =\> ["log\_timestamp", "YYYY-MM-dd HH:mm:ss"]  
timezone =\> "Etc/UTC"  
}

useragent {  
source=\> "useragent"  
prefix=\> "browser"  
}

mutate {  
remove\_field =\> ["log\_timestamp"]  
}  
if [type] == "iisbeat" and [response] not in ["404", "444"] {  
drop {}  
}

if [type] == "wineventlog"  
{

if [event\_id] not in [4625,4666] {  
drop {}  
}  
}

if [type] == "apachebeat"  
{

grok {  
match =\> { "message" =\> "%{COMMONAPACHELOG}" }  
add\_tag =\> ["apachebeat", "grokked"]  
}

}  
if [type] == "apachebeat" and [response] not in ["404", "403"] {  
drop {}  
}

}  
else if [tags] not in ["grokked"]  
{drop{}}  
}

# filter configuration for another machines

filter  
{  
if [host] == "datanode3"  
{  
if [type] == "log"  
{  
grok  
{  
match =\> ["message", "%{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}: %{WORD:authentication}"]  
add\_tag =\> ["log", "grokked"]  
}  
}

if [type] == "log" and [authentication] not in ["Failed","info"]  
{  
drop{}  
}

if [type] == "iisbeat"  
{  
#ignore log comments  
if [message] =~ "^#" {  
drop {}  
}

grok {  
# check that fields match your IIS log settings  
match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{IPORHOST:hostip} %{WORD:method} %{NOTSPACE:page} %{NOTSPACE:query} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clientip} %{NOTSPACE:useragent} %{NOTSPACE:referrer} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:scstatus} %{NUMBER:timetaken}"]  
}  
}

#Set the Event Timesteamp from the log  
date {  
match =\> ["log\_timestamp", "YYYY-MM-dd HH:mm:ss"]  
timezone =\> "Etc/UTC"  
}

useragent {  
source=\> "useragent"  
prefix=\> "browser"  
}

mutate {  
remove\_field =\> ["log\_timestamp"]  
}  
if [type] == "iisbeat" and [response] not in ["404", "444"] {  
drop {}  
}

if [type] == "wineventlog"  
{

if [event\_id] not in [4625,4666] {  
drop {}  
}  
}

if [type] == "apachebeat"  
{

grok {  
match =\> { "message" =\> "%{COMMONAPACHELOG}" }  
add\_tag =\> ["apachebeat", "grokked"]  
}

}  
if [type] == "apachebeat" and [response] not in ["404", "403"] {  
drop {}  
}

}  
else if [tags] not in ["grokked"]  
{drop{}}

}

---

<div class="post-metadata">

**Author:** ![Muhasin\_Cm](https://avatars.discourse-cdn.com/v4/letter/m/a6a055/32.png) [@Muhasin\_Cm](https://discuss.elastic.co/u/Muhasin_Cm)\
**Post date:** [May 5, 2016, 7:34am UTC](https://discuss.elastic.co/t/multiple-filter-files-for-different-host/49262/4 "2016-05-05T07:34:06Z")

</div>

I have seperate input and output files inside the logstash conf directory.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:59am UTC](https://discuss.elastic.co/t/multiple-filter-files-for-different-host/49262/5 "2017-07-06T04:59:20Z")

</div>