# Location of custom filters

**URL:** <https://discuss.elastic.co/t/location-of-custom-filters/24700>\
**Category:** Logstash\
**Created:** [July 1, 2015, 9:28am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700 "2015-07-01T09:28:53Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![sushmithak](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@sushmithak](https://discuss.elastic.co/u/sushmithak)\
**Post date:** [July 1, 2015, 9:28am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/1 "2015-07-01T09:28:53Z")

</div>

Hi,

I want to extract required fields from syslog-messages where can i write (exact location) custom filters in logstash.  
plz provide any examples to write custom filters.  
-thanks

---

<div class="post-metadata">

**Author:** ![praveench](https://avatars.discourse-cdn.com/v4/letter/p/f9ae1b/32.png) [@praveench](https://discuss.elastic.co/u/praveench)\
**Post date:** [July 1, 2015, 9:30am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/2 "2015-07-01T09:30:17Z")

</div>

Hi,

I am also looking for that...

Please any one suggest us..

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 1, 2015, 9:40am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/3 "2015-07-01T09:40:17Z")

</div>

Logstash is normally configured to read all configuration files from /etc/logstash/conf.d in alphabetical order. You could e.g. create a syslog.conf file and put all your syslog-related filters there.

However, keep in mind that unless you wrap filters in explicit conditionals they apply to all messages. You don't want your syslog grok filter to attempt to parse your httpd logs, for example.

---

<div class="post-metadata">

**Author:** ![sushmithak](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@sushmithak](https://discuss.elastic.co/u/sushmithak)\
**Post date:** [July 1, 2015, 9:53am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/4 "2015-07-01T09:53:36Z")

</div>

Thankyou Magnus Bäck.

here is my 10-syslog.conf file

filter {  
if [type] == "syslog" {  
grok {

```
 match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
  add_field => ["received_at", "%{@timestamp}"]
  add_field => ["received_from", "%{host}"]
}
syslog_pri { }
date {
  match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
}

```

}  
}  
i want to extract fields in syslog\_message.  
plz tell me a way to do it.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 1, 2015, 10:07am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/5 "2015-07-01T10:07:46Z")

</div>

You could use grok for that too, but it depends on what the message looks like. Please provide examples.

---

<div class="post-metadata">

**Author:** ![sushmithak](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@sushmithak](https://discuss.elastic.co/u/sushmithak)\
**Post date:** [July 1, 2015, 10:28am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/6 "2015-07-01T10:28:49Z")

</div>

here is the example for syslog\_message.

i have added another grok to my 10-syslog.conf file like this:

filter {  
if [type] == "syslog" {  
grok {  
match =\> ["message" , "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" ]  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
}  
}

```
syslog_pri { }
date {
  match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
}

```

}  
}

the pattern is working correctly in grok debugger.  
the problem is fields are not getting extracted and not shown in kibana .  
adding my grok filter in 10-syslog.conf is correct ??

---

<div class="post-metadata">

**Author:** ![sushmithak](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@sushmithak](https://discuss.elastic.co/u/sushmithak)\
**Post date:** [July 1, 2015, 11:51am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/7 "2015-07-01T11:51:59Z")

</div>

any suggestions plz...

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 1, 2015, 11:59am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/8 "2015-07-01T11:59:23Z")

</div>

Yeah, that should work. Are you getting the `_grokparsefailure` tag for those messages?

For parsing this particular kind of message the [kv filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html) might be more convenient.

---

<div class="post-metadata">

**Author:** ![sushmithak](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@sushmithak](https://discuss.elastic.co/u/sushmithak)\
**Post date:** [July 1, 2015, 12:07pm UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/9 "2015-07-01T12:07:21Z")

</div>

Thanks magnus,  
I'm not getting \_grokparsefailure tag.  
i have changed the 10-syslog.conf file as shown above and restarted the logstash.  
but kibana is showing the default fields only.

---

<div class="post-metadata">

**Author:** ![sushmithak](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@sushmithak](https://discuss.elastic.co/u/sushmithak)\
**Post date:** [July 1, 2015, 12:19pm UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/10 "2015-07-01T12:19:50Z")

</div>

hi  
Thanks

---

<div class="post-metadata">

**Author:** ![sushmithak](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@sushmithak](https://discuss.elastic.co/u/sushmithak)\
**Post date:** [July 2, 2015, 6:01am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/11 "2015-07-02T06:01:05Z")

</div>

Extracted fields are not displaying in kibana.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:35am UTC](https://discuss.elastic.co/t/location-of-custom-filters/24700/12 "2017-07-06T05:35:46Z")

</div>