# Grok does not work

**URL:** <https://discuss.elastic.co/t/grok-does-not-work/85995>\
**Category:** Logstash\
**Created:** [May 16, 2017, 5:26pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995 "2017-05-16T17:26:12Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Igor\_Gerasimow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_gerasimow/32/18258_2.png) [@Igor\_Gerasimow](https://discuss.elastic.co/u/Igor_Gerasimow)\
**Post date:** [May 16, 2017, 5:26pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/1 "2017-05-16T17:26:12Z")

</div>

Hello,  
i had such log row ( for example ) `10.128.0.23 - - [16/May/2017:12:24:16 -0000] "GET /pp/api/personalprofile/update_status/na/Sweaty%20ASol HTTP/1.1" 101 361"`

- and I use such grok for parsing

`%{IP:clientip} \- \- \[%{HTTPDATE:timestamp}\] "%{WORD:action} /%{GREEDYDATA:message} %{WORD:protocol}/%{NUMBER:protocolNum}" %{NUMBER:status} %{NUMBER}`

Whore filter

```auto
filter {
grok{
match=>{
"message" => "%{IP:clientip} \- \- \[%{NOTSPACE:date} \+%{INT}\] \"%{WORD:action} /%{WORD}/%{WORD}/%{NOTSPACE:verb} %{WORD:protocol}/%{NUMBER:protocolNum}\" %{NUMBER:status} %{NUMBER}"
}
add_field=>{
"eventName"=>"grok"
}
}
geoip {
source => "clientip"
}
}

```

I checked this grok with [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/) and it works fine  
However i getting such error on Logstash 5.4  
`[2017-05-16T17:11:29,774][ERROR][logstash.agent] Cannot create pipeline {:reason=>"Expected one of #, {, } at line 4, column 63 (byte 87) after filter {\ngrok{\nmatch=>{\n\"log\" => \"%{IP:clientip} \\- \\- \\[%{NOTSPACE:date} \\+%{INT}\\] \""}`

Please advice - what i 'm doing wrong ?  
Thank you in advance.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [May 16, 2017, 5:38pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/2 "2017-05-16T17:38:29Z")

</div>

Why not use the built-in `COMMONAPACHELOG` grok pattern?

```auto
filter {
  grok { match => { "message" => "%{COMMONAPACHELOG}" } }
  date { match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"] }
}

```

I put a simple configuration together and it works:

```auto
input { stdin {} }

filter {
  grok { match => { "message" => "%{COMMONAPACHELOG}" } }
  date { match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"] }
}

output { stdout { codec => rubydebug } }

```

Here's what it looks like when I paste your sample from above:

```auto
[2017-05-16T11:36:23,922][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9601}
10.128.0.23 - - [16/May/2017:12:24:16 -0000] "GET /pp/api/personalprofile/update_status/na/Sweaty%20ASol HTTP/1.1" 101 361
{
        "request" => "/pp/api/personalprofile/update_status/na/Sweaty%20ASol",
           "auth" => "-",
          "ident" => "-",
           "verb" => "GET",
        "message" => "10.128.0.23 - - [16/May/2017:12:24:16 -0000] \"GET /pp/api/personalprofile/update_status/na/Sweaty%20ASol HTTP/1.1\" 101 361",
     "@timestamp" => 2017-05-16T12:24:16.000Z,
       "response" => "101",
          "bytes" => "361",
       "clientip" => "10.128.0.23",
       "@version" => "1",
           "host" => "localhost.local",
    "httpversion" => "1.1",
      "timestamp" => "16/May/2017:12:24:16 -0000"
}

```

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [May 16, 2017, 5:43pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/3 "2017-05-16T17:43:11Z")

</div>

As an aside, this is how that pattern is built:

```auto
COMMONAPACHELOG %{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)

```

You can see how each of these meta-patterns are built in the [github repository](https://github.com/logstash-plugins/logstash-patterns-core/blob/v4.1.0/patterns/grok-patterns).

---

<div class="post-metadata">

**Author:** ![jovanmal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jovanmal/32/17330_2.png) [@jovanmal](https://discuss.elastic.co/u/jovanmal)\
**Post date:** [May 16, 2017, 5:55pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/4 "2017-05-16T17:55:43Z")

</div>

Hi,

@Igor_Gerasimow

did you tried to use single quotes instead of double ons in message expression? So, instead of

> [@Igor\_Gerasimow](#):
>
> "message" =\> "%{IP:clientip} - - [%{NOTSPACE:date} +%{INT}] "%{WORD:action} /%{WORD}/%{WORD}/%{NOTSPACE:verb} %{WORD:protocol}/%{NUMBER:protocolNum}" %{NUMBER:status} %{NUMBER}"

should be

> "message" =\> '%{IP:clientip} - - [%{NOTSPACE:date} +%{INT}] "%{WORD:action} /%{WORD}/%{WORD}/%{NOTSPACE:verb} %{WORD:protocol}/%{NUMBER:protocolNum}" %{NUMBER:status} %{NUMBER}'

---

<div class="post-metadata">

**Author:** ![Igor\_Gerasimow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_gerasimow/32/18258_2.png) [@Igor\_Gerasimow](https://discuss.elastic.co/u/Igor_Gerasimow)\
**Post date:** [May 16, 2017, 6:39pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/5 "2017-05-16T18:39:21Z")

</div>

Well - it is nginx web server log - is it will be work ?  
BTW - not it is impossible to change log format.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [May 16, 2017, 6:56pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/6 "2017-05-16T18:56:41Z")

</div>

nginx uses the same basic format.

---

<div class="post-metadata">

**Author:** ![Igor\_Gerasimow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_gerasimow/32/18258_2.png) [@Igor\_Gerasimow](https://discuss.elastic.co/u/Igor_Gerasimow)\
**Post date:** [May 16, 2017, 9:26pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/7 "2017-05-16T21:26:00Z")

</div>

Hi - no this not gonna work because in my log could be `[16/May/2017:12:24:16 -0000]`  
and your example dill drop it, because of  
`"message" => '%{IP:clientip} - - [%{NOTSPACE:date} +%{INT}] \"`

---

<div class="post-metadata">

**Author:** ![jovanmal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jovanmal/32/17330_2.png) [@jovanmal](https://discuss.elastic.co/u/jovanmal)\
**Post date:** [May 16, 2017, 9:32pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/8 "2017-05-16T21:32:34Z")

</div>

Can you post output of command

> /usr/share/logstash/bin/logstash --path.config /etc/logstash/your\_logstash\_config.conf --config.test\_and\_exit

Change path to one that corresponds to your config file, of course

---

<div class="post-metadata">

**Author:** ![Igor\_Gerasimow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_gerasimow/32/18258_2.png) [@Igor\_Gerasimow](https://discuss.elastic.co/u/Igor_Gerasimow)\
**Post date:** [May 18, 2017, 2:32pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/9 "2017-05-18T14:32:26Z")

</div>

> [@jovanmal](#):
>
> --config.test\_and\_exit

i did it.

```auto
/usr/share/logstash/bin/logstash --path.settings /etc/logstash/ --config.test_and_exit

Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties

```

---

<div class="post-metadata">

**Author:** ![jovanmal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jovanmal/32/17330_2.png) [@jovanmal](https://discuss.elastic.co/u/jovanmal)\
**Post date:** [May 19, 2017, 6:05am UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/10 "2017-05-19T06:05:00Z")

</div>

You didn't specified exact config file, you have to do it.

See results in Logstash log file and post it

---

<div class="post-metadata">

**Author:** ![Igor\_Gerasimow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_gerasimow/32/18258_2.png) [@Igor\_Gerasimow](https://discuss.elastic.co/u/Igor_Gerasimow)\
**Post date:** [May 19, 2017, 1:37pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/11 "2017-05-19T13:37:50Z")

</div>

Hi - you right 🙂

```auto
10.128.0.17 - - [19/May/2017:12:29:12 +0000] "GET /public/a245afb093cb3064f1909c02782cbc63.jpg HTTP/1.1" 200 175003 "https://site.domain.st/profile/euw/BornToDieftw" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" "-"
{
        "request" => "/public/a245afb093cb3064f1909c02782cbc63.jpg",
          "agent" => "\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36\"",
           "auth" => "-",
          "ident" => "-",
           "verb" => "GET",
        "message" => "10.128.0.17 - - [19/May/2017:12:29:12 +0000] \"GET /public/a245afb093cb3064f1909c02782cbc63.jpg HTTP/1.1\" 200 175003 \"https://site.domain.st/profile/euw/BornToDieftw\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36\" \"-\"",
       "referrer" => "\"https://stats.mobalytics.gg/profile/euw/BornToDieftw\"",
     "@timestamp" => 2017-05-19T12:29:12.000Z,
       "response" => "200",
          "bytes" => "175003",
       "clientip" => "10.128.0.17",
       "@version" => "1",
           "host" => "elasticsearch-logs",
    "httpversion" => "1.1",
      "timestamp" => "19/May/2017:12:29:12 +0000"
}

```

but in kibana i still does not see parsing result ☹

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 16, 2017, 1:38pm UTC](https://discuss.elastic.co/t/grok-does-not-work/85995/12 "2017-06-16T13:38:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.