# Grok-Filter not working **URL:** \ **Category:** Logstash\ **Created:** [May 8, 2018, 1:06pm UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009 "2018-05-08T13:06:50Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Dante91](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dante91](https://discuss.elastic.co/u/Dante91)\ **Post date:** [May 8, 2018, 1:06pm UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/1 "2018-05-08T13:06:50Z")
Hey, I want to log my MikroTik-Router. Thatfor I get for example the following log: > May/03/2018 17:37:01 system,info,account user admin logged out from 192.168.88.1 via telnet With help from the grok debugger at [grokdebug.herokuapp.com](http://grokdebug.herokuapp.com) I get the following config: > input { > tcp { > port =\> 5514 > type =\> "MikroTik" > } > udp { > port =\> 5514 > type =\> "MikroTik" > } > } > filter { > if [type] == "MikroTik" { > grok { > match =\> {"message" =\> "(?"\<"logtimestamp"\>"%{MONTH}/%{MONTHDAY}/%{YEAR} %{TIME}) %{WORD:source},%{WORD:level},%{GREEDYDATA:message}"} > } > mutate { > add\_tag =\> ["grokked"] > add\_field =\> { > "source" =\> "%{source)" > "level" =\> "%(level)" > } > } > } > } > output { > stdout { codec =\> rubydebug } > } I had to do the brackets around the "logtimestamp" because of the formatting. When I start logstash with this config I get following output: > { > "host" =\> "192.168.88.3", > "message" =\> "system,info,account user admin logged out from 192.168.88.1 via telnet", > "level" =\> "%(level)", > "source" =\> "%{source)", > "@version" =\> "1", > "type" =\> "MikroTik", > "@timestamp" =\> 2018-05-08T02:44:38.508Z, > "tags" =\> [ > [0] "\_grokparsefailure", > [1] "grokked" > ] > } Unfortunately the grok-filter doesn´t work for me. My logstash-version is the 6.2.3 Hope someone can help me debugging my config. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 8, 2018, 1:49pm UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/2 "2018-05-08T13:49:40Z")
Your configuration isn't consistent with the output you say you're getting. If the grok filter indeed wasn't working, how come the timestamp is stripped away from the `message` field? ---
**Author:** ![Dante91](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dante91](https://discuss.elastic.co/u/Dante91)\ **Post date:** [May 8, 2018, 2:33pm UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/3 "2018-05-08T14:33:33Z")
Hey magnusbaeck, thanks for the quick reply. Where could this timestamp come from? I start logstash with my testconfig.conf and following parameters: > ./logstash -f testconfig.conf --path.settings /etc/logstash/ The conf.d-folder in /etc/logstash is empty and my logstash folder is in /usr/share/logstash/bin. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 8, 2018, 3:05pm UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/4 "2018-05-08T15:05:42Z")
```auto grok { match => {"message" => "(?%{MONTH}/%{MONTHDAY}/%{YEAR} %{TIME}) %{WORD:source},%{WORD:level},%{GREEDYDATA:message}"} } ``` will match that message. What are you trying to do with the mutate+add\_field? ```auto "source" => "%{source)" ``` You have mis-matched braces and parentheses. If we correct that, the source field already exists, so adding a second source field just forces it to be an array. Similarly for grokking a message field using a regexp that has an item called message -- message ends up as an array, which is probably not what you want. ```auto "message" => [ [0] "May/03/2018 17:37:01 system,info,account user admin logged out from 192.168.88.1 via telnet", [1] "account user admin logged out from 192.168.88.1 via telnet" ], ``` ---
**Author:** ![Dante91](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dante91](https://discuss.elastic.co/u/Dante91)\ **Post date:** [May 15, 2018, 8:53am UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/5 "2018-05-15T08:53:25Z")
Hey Badger, to eliminate configuration misstakes, I installed a completely new machine with the actual versions of elk-stack. In my filter-options I only placed the grok-filter without any other options and renamed the fields: > filter { > if [type] == "MikroTik" { > grok { > match =\> {"test012" =\> "(?%{MONTH}/%{MONTHDAY}/%{YEAR} %{TIME}) %{WORD:test123},%{GREEDYDATA:test234}"} > } > } > } So there should be no field that is already "in use" by logstash. Unfortunately I don´t even get the hoped result: > { > "tags" =\> [ > [0] "\_grokparsefailure" > ], > "message" =\> "system,info,account user admin logged in from 192.168.88.1 via telnet", > "@version" =\> "1", > "type" =\> "MikroTik", > "host" =\> "192.168.88.3", > "@timestamp" =\> 2018-05-09T08:32:20.308Z > } Again there is this grokparsefailure. Isn´t there any log that could explain where this error comes from? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 15, 2018, 9:14am UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/6 "2018-05-15T09:14:31Z")
The grok filter fails because your event doesn't have a `test012` field. ---
**Author:** ![Dante91](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dante91](https://discuss.elastic.co/u/Dante91)\ **Post date:** [May 15, 2018, 10:06am UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/7 "2018-05-15T10:06:37Z")
Thanks for the quick reply, even if I change the test012-fielt into "message" the output doesn´t change. > { > "@version" =\> "1", > "@timestamp" =\> 2018-05-09T09:40:53.305Z, > "host" =\> "192.168.88.3", > "tags" =\> [ > [0] "\_grokparsefailure" > ], > "type" =\> "MikroTik", > "message" =\> "system,info,account user admin logged in from 192.168.88.1 via telnet" > } Again the "\_grokparsefailure"-tag ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 15, 2018, 11:19am UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/8 "2018-05-15T11:19:42Z")
`message` obviously doesn't match the grok expression. ---
**Author:** ![Dante91](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dante91](https://discuss.elastic.co/u/Dante91)\ **Post date:** [May 15, 2018, 11:32am UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/9 "2018-05-15T11:32:28Z")
Then I think that I understood something wrong. In my output there are standard defined variables like "@version", "host", "message", .... The type-tag was given by logstash on the input-conf. Now I want the grok filter to seperate this field "message" into seperate fields, in my case "test123" and "test234". What did I understand wrong? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 15, 2018, 1:29pm UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/10 "2018-05-15T13:29:55Z")
> [@Dante91](#): > > Now I want the grok filter to seperate this field "message" into seperate fields, in my case "test123" and "test234". > > What did I understand wrong? In the grok filter you said it should match a date, followed by a word, followed by the rest of the line. But your message does not have a date, so grok is going to fail. ---
**Author:** ![Dante91](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dante91](https://discuss.elastic.co/u/Dante91)\ **Post date:** [May 16, 2018, 5:33am UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/11 "2018-05-16T05:33:22Z")
Hey badger, that was the solution, I didn´t notice that the timestamp wasn´t already in my message output. Thanks to you and magnusbaeck. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 13, 2018, 5:33am UTC](https://discuss.elastic.co/t/grok-filter-not-working/131009/12 "2018-06-13T05:33:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.