# Logstash Filters **URL:** \ **Category:** Logstash\ **Created:** [July 9, 2015, 6:06am UTC](https://discuss.elastic.co/t/logstash-filters/25203 "2015-07-09T06:06:06Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![abathula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abathula/32/3595_2.png) [@abathula](https://discuss.elastic.co/u/abathula)\ **Post date:** [July 9, 2015, 6:06am UTC](https://discuss.elastic.co/t/logstash-filters/25203/1 "2015-07-09T06:06:06Z")
Hi Good morning, I have a requirement of creating a filter in logstash which is displayed as a field in kibana also. if any ( **Exception** or **IOException** or **NullPointerException** ) in message, then create a field called **msg\_exception** in logstash. **find my configuration:** filter{ grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:time} [%{NUMBER:thread}] %{LOGLEVEL:loglevel} %{GREEDYDATA:class} -%{GREEDYDATA:msg} " } } if "Exception" in [msg] { grok { add\_field =\> { "msg\_error" =\> "%{msg}" } } } } Problem is data is loaded into **ES** through **Logstash** , but am not getting any filter. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 9, 2015, 6:17am UTC](https://discuss.elastic.co/t/logstash-filters/25203/2 "2015-07-09T06:17:22Z")
When you have an `add_field` declaration in a filter it means "add the fields if the filter is successful", but in your second grok filter you're specifying any matches criteria so the filter probably doesn't consider itself successful. Use a [mutate filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html) instead: ``` mutate { add_field => { "msg_error" => "%{msg}" } } ``` Also, avoid using multiple GREEDYDATA patterns. In this case you're probably okay most of the time but it can really mess things up if one isn't careful. In this case I suggest using `%{JAVACLASS:class}` or `%{NOTSPACE:class}` instead (and consider naming the field e.g. `logger` since that describes better what the field actually contains). ---
**Author:** ![abathula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abathula/32/3595_2.png) [@abathula](https://discuss.elastic.co/u/abathula)\ **Post date:** [July 9, 2015, 7:00am UTC](https://discuss.elastic.co/t/logstash-filters/25203/3 "2015-07-09T07:00:01Z")
@magnusbaeck, here am getting the problem like all records are displaying when click on **msg\_error** , But i need only where **Exception| IOException| NullPointerException** is matching that record need to be filtered in kibana. **Problem in Kibana:** ![](https://us1.discourse-cdn.com/elastic/original/2X/e/efd9948e00cd893f9f0f6c7ad20bb865c2ce63ee.png) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 9, 2015, 7:06am UTC](https://discuss.elastic.co/t/logstash-filters/25203/4 "2015-07-09T07:06:40Z")
You want to list only messages whose `msg_error` field is set? In that case, add e.g. `AND msg_error:*` to the query. ---
**Author:** ![abathula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abathula/32/3595_2.png) [@abathula](https://discuss.elastic.co/u/abathula)\ **Post date:** [July 9, 2015, 7:15am UTC](https://discuss.elastic.co/t/logstash-filters/25203/5 "2015-07-09T07:15:29Z")
I am writing the filter like this, filter{ grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:time} [%{NUMBER:thread}] %{LOGLEVEL:loglevel} %{NOTSPACE:logger} -%{GREEDYDATA:msg} " } } if "Exception" in [msg] { grok { add\_field =\> { "msg\_error" =\> "%{msg}AND msg\_error:\*" } } } } Is it right ?? i am confusing where to write the query ... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 9, 2015, 7:25am UTC](https://discuss.elastic.co/t/logstash-filters/25203/6 "2015-07-09T07:25:59Z")
The _Kibana_ query. If, as I said, the problem is that seeing all kinds of events and not just those with the `msg_error` field set. Also, you're still using the grok filter even though I suggest that you use a mutate filter. That _might_ work but it doesn't look right. ---
**Author:** ![abathula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abathula/32/3595_2.png) [@abathula](https://discuss.elastic.co/u/abathula)\ **Post date:** [July 9, 2015, 8:32am UTC](https://discuss.elastic.co/t/logstash-filters/25203/7 "2015-07-09T08:32:15Z")
Ya right, it is working when the word is only **Exception**. but i need the solution for **IOException** and **NullPointerException** (search with the Exception, IOException and other exception are not searched) Here i am getting am getting a new problem (-grokparsefailure) when this type of log is loading into **ES** **Actually my log is:** ![](https://us1.discourse-cdn.com/elastic/original/2X/1/195ee433ec8faf2d80aaedaa8d2d871d3144efd6.png) **Pattern is** "%{TIMESTAMP\_ISO8601:time} [%{NUMBER:thread}] %{LOGLEVEL:loglevel} %{NOTSPACE:logger} -%{GREEDYDATA:msg} " } -\> is there any need to add the new pattern in filter ?? **existing filter configuration is:** filter{ grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:time} [%{NUMBER:thread}] %{LOGLEVEL:loglevel} %{GREEDYDATA:class} -%{GREEDYDATA:msg} " } } if "Exception" in [msg] { mutate { add\_field =\> { "msg\_error" =\> "%{msg}" } } } } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 9, 2015, 10:05am UTC](https://discuss.elastic.co/t/logstash-filters/25203/8 "2015-07-09T10:05:19Z")
> Ya right, it is working when the word is only Exception. but i need the solution for IOException and NullPointerException(search with the Exception, IOException and other exception are not searched) Whether it's just Exception or IOException doesn't matter. `if "Exception" in [msg]` is a "stupid" substring search that doesn't care about word boundaries. > "%{TIMESTAMP\_ISO8601:time} [%{NUMBER:thread}] %{LOGLEVEL:loglevel} > %{NOTSPACE:logger} -%{GREEDYDATA:msg} " } > -\> is there any need to add the new pattern in filter ?? There's a trailing space in your pattern that shouldn't be there. Also, square brackets are metacharacters so `[%{NUMBER:thread}]` must be `\[%{NUMBER:thread}\]`. Next time please don't post screenshots. Use the copy/paste feature that I'm sure you're you're familiar with. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:35am UTC](https://discuss.elastic.co/t/logstash-filters/25203/9 "2017-07-06T05:35:13Z")