# Problem parsing a datestring - Grok Debugger shows no problem

**URL:** <https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639>\
**Category:** Logstash\
**Created:** [November 21, 2018, 7:36am UTC](https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639 "2018-11-21T07:36:42Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![faulander](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faulander/32/36344_2.png) [@faulander](https://discuss.elastic.co/u/faulander)\
**Post date:** [November 21, 2018, 7:36am UTC](https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639/1 "2018-11-21T07:36:42Z")

</div>

Example Logfile Entry which works perfectly in the Grok Debugger:

> `2018-11-21 08:15:54,809 WARN [com.xxx.mediaserver.tools.maintenance.FileMaintenance] (Thread-6) Not removing directory C:\xxx\mediaserver\import_export\SUPERVISOR\Downloads\packaged_ko_40186264_40186283 because we're not sure if it was created by us (directory name mismatch).`

Here's the identification string:

> `%{DATE:date}%{SPACE}%{TIME:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}\[(?<class>.+?(?=]))\]%{SPACE}\((?<thread>.+?(?=\)))\)%{SPACE}(?<logmessage>.*)`

And here's the logstash part:

> grok { match =\> { "message" =\> "%{DATE:date}%{SPACE}%{TIME:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}[(?.+?(?=]))]%{SPACE}((?.+?(?=))))%{SPACE}(?.\*)" } }  
> mutate { add\_field =\> { "access\_time" =\> "%{date} %{timestamp}" } }  
> date { match =\> ["access\_time", "YY-MM-dd HH:mm:ss,SSS"] }  
> mutate {  
> remove\_field =\> ["SPACE"]  
> remove\_tag =\> ["beats\_input\_codec\_plain\_applied", "monitor", "apache"]  
> add\_tag =\> ["mediaserver"]

In the field access\_time i have this:

> %{month} %{day} %{year} %{hour}:%{minute}:%{seconds}

so it seems, date parsing fails.

I also have this in field thread which makes no sense:

> `Thread-6) Not removing directory C:\xxx\mediaserver\import_export\SUPERVISOR\Downloads\packaged_ko_40186264_40186283 because we're not sure if it was created by us (directory name mismatch`

because my regexp is parsing for the finishing ")" which works perfectly in Grok Debugger and ALSO in another part of the logstash config, which i post below:

This works perfectly:

> ```
> else if [quelle] == "wildfly-main" {
> grok { match => { "message" => "%{DATE:date}%{SPACE}%{TIME:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}\[(?<class>.+?(?=:)):%{INT:linenumber}\]%{SPACE}\((?<thread>.+?(?=\)))\)%{SPACE}(?<logmessage>.*)" } }
> mutate { add_field => { "access_time" => "%{date} %{timestamp}" } }
> date { match => ["access_time", "YY-MM-dd HH:mm:ss,SSS"] }
> mutate { 
> remove_field => ["SPACE", "date", "timestamp", "message"] 
> remove_tag => ["beats_input_codec_plain_applied", "monitor", "apache"]
> add_tag => ["wildfly"]
> }
> }
> 
> ```

---

<div class="post-metadata">

**Author:** ![Eniqmatic](https://avatars.discourse-cdn.com/v4/letter/e/ea5d25/32.png) [@Eniqmatic](https://discuss.elastic.co/u/Eniqmatic)\
**Post date:** [November 21, 2018, 2:03pm UTC](https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639/2 "2018-11-21T14:03:02Z")

</div>

> [@faulander](#):
>
> %{DATE:date}%{SPACE}%{TIME:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}[(?\<class\>.+?(?=]))]%{SPACE}((?\<thread\>.+?(?=))))%{SPACE}(?\<logmessage\>.\*)

Any reason that this part is different to this:

> [@faulander](#):
>
> And here's the logstash part:
> 
> > grok { match =\> { "message" =\> "%{DATE:date}%{SPACE}%{TIME:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}[(?.+?(?=]))]%{SPACE}((?.+?(?=))))%{SPACE}(?.\*)" } }

---

<div class="post-metadata">

**Author:** ![faulander](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faulander/32/36344_2.png) [@faulander](https://discuss.elastic.co/u/faulander)\
**Post date:** [November 21, 2018, 2:20pm UTC](https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639/3 "2018-11-21T14:20:47Z")

</div>

copy&paste error. this is the correct part in the logstash-config:

> ```
> else if [quelle] == "mediaserver-main" {
> grok { match => { "message" => "%{DATE:date}%{SPACE}%{TIME:timestamp}%{SPACE}%{WORD:loglevel}%{SPACE}\[(?<class>.+?(?=]))\]%{SPACE}\((?<thread>.+?(?=\)))\)%{SPACE}(?<logmessage>.*)" } }
> mutate { add_field => { "access_time" => "%{date} %{timestamp}" } }
> date { match => ["access_time", "YY-MM-dd HH:mm:ss,SSS"] }
> mutate { 
> remove_field => ["SPACE"] 
> remove_tag => ["beats_input_codec_plain_applied", "monitor", "apache"]
> add_tag => ["mediaserver"]
> }
> }
> 
> ```

So yes, i double and tripple checked that the correct string (which works in Grok Debugger) is also in the Logstash-Config. I am unable to find out why it doesn't work. I have another else if block with almost the exact same string which works like explained ☹

---

<div class="post-metadata">

**Author:** ![Eniqmatic](https://avatars.discourse-cdn.com/v4/letter/e/ea5d25/32.png) [@Eniqmatic](https://discuss.elastic.co/u/Eniqmatic)\
**Post date:** [November 21, 2018, 3:58pm UTC](https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639/4 "2018-11-21T15:58:56Z")

</div>

Testing your Grok only it works for me:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/5/e507b75d78a8d1cb0f8e6a9fbdd11037e3c370ad.png)

I will try the rest of the filter tomorrow when I have more time.

---

<div class="post-metadata">

**Author:** ![faulander](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faulander/32/36344_2.png) [@faulander](https://discuss.elastic.co/u/faulander)\
**Post date:** [November 22, 2018, 7:59am UTC](https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639/5 "2018-11-22T07:59:22Z")

</div>

yes, that's the funny thing. The second part in the logstash almost parses the same and it works aswell. I don't get it, try to solve it since 2-3 days.

What i did additionally:

- Completely deleted indices in Elastic
- Rebuild fields in Elastic
- Reconfigured positioning in Logstash

Still the same problem 😕

---

<div class="post-metadata">

**Author:** ![faulander](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faulander/32/36344_2.png) [@faulander](https://discuss.elastic.co/u/faulander)\
**Post date:** [November 23, 2018, 10:05am UTC](https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639/6 "2018-11-23T10:05:37Z")

</div>

fixed the problem myself.  
And now comes the funny part:

I just changed the position of the "else if" block in the logstash config. Nothing else.  
Since then it works like expected.

I even copied it back to the original position (last one) and it still works.

Sometimes strange things happen and resolve themselves ...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 21, 2018, 10:05am UTC](https://discuss.elastic.co/t/problem-parsing-a-datestring-grok-debugger-shows-no-problem/157639/7 "2018-12-21T10:05:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.