# Grok parsing in logstash for Windows DNS Debug Log **URL:** \ **Category:** Logstash\ **Created:** [December 6, 2021, 9:25pm UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085 "2021-12-06T21:25:47Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![craign30](https://avatars.discourse-cdn.com/v4/letter/c/97f17d/32.png) [@craign30](https://discuss.elastic.co/u/craign30)\ **Post date:** [December 6, 2021, 9:25pm UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/1 "2021-12-06T21:25:47Z")
I'm using grok in Logstash (7.8.0) to parse data from a Windows Server (2019) DNS debug log (sent via filebeat) using the statement below. Most of the time, the data gets parsed correctly and fields are populated and visible in Kibana. However, roughly 1/3 of the time, the data doesn't appear to get parsed and the fields aren't visible in Kibana. In this scenario, I know the data is getting from Logstash to Elasticsearch because the message field in Kibana is populated with the entry from the DNS debug log. I've confirmed that the pattern is the same between data that does and doesn't correctly get parsed. In some cases, there may be 10-20 entries with the same timestamp (to the second) in the DNS log. Is this potentially a situation where Logstash / grok just can't keep up? ```auto grok { match => { "message" => [ "(?%{DATE_US} %{TIME} (?:AM|PM))\s+%{NUMBER}\s+%{WORD:t1}\s+%{BASE16NUM}\s+%{WORD:network.transport}\s+%{WORD:network.direction}\s+%{IP:dns.resolved_ip}\s+%{BASE16NUM}\s+%{WORD:dns.op_code}\s+\[%{BASE16NUM}\s+%{WORD:dns.header_flags}\s+%{WORD:dns.response_code}\]\s+%{WORD:dns.question.type}\s+%{GREEDYDATA:dns.question.name}" ] } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [December 6, 2021, 9:36pm UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/2 "2021-12-06T21:36:16Z")
> [@craign30](#): > > Is this potentially a situation where Logstash / grok just can't keep up? I very much doubt it. Do the unparsed events have a \_grokparsefailure tag? Maybe WORD doesn't match your data. WORD is `\b\w+\b` and \w is `[a-zA-Z0-9_]` so it accepts underscore but not hyphen or period. ---
**Author:** ![craign30](https://avatars.discourse-cdn.com/v4/letter/c/97f17d/32.png) [@craign30](https://discuss.elastic.co/u/craign30)\ **Post date:** [December 6, 2021, 9:55pm UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/3 "2021-12-06T21:55:24Z")
> [@Badger](#): > > \_grokparsefailure Hi Badger. Would the \_grokparsefailure be visible in Kibana? If so, it's not there for the events in question. Below is a sample log entry. All fields with the WORD designation are text. Thanks 12/6/2021 2:54:31 PM 2518 PACKET 00000230C7FC09D0 UDP Rcv 192.168.1.1 2be4 Q [0001 D NOERROR] A (7)outlook(2)ha(9)office365(3)com(0) ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [December 6, 2021, 10:03pm UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/4 "2021-12-06T22:03:59Z")
The tags field should certainly be visible in kibana. That grok pattern matchs that message for me, so I have no idea what could be going wrong. ---
**Author:** ![craign30](https://avatars.discourse-cdn.com/v4/letter/c/97f17d/32.png) [@craign30](https://discuss.elastic.co/u/craign30)\ **Post date:** [December 6, 2021, 10:11pm UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/5 "2021-12-06T22:11:49Z")
Yea, it seems to skip the parsing process for certain lines all together, as the fields (dns.resolved\_ip, etc.) don't even exist in Kibana for those entries. Also, only the dns value exists in the tags field, which gets applied from filebeat. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [December 6, 2021, 10:24pm UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/6 "2021-12-06T22:24:42Z")
Is it possible that one of your beats is sending directly to Elasticsearch and bypassing logstash? ---
**Author:** ![craign30](https://avatars.discourse-cdn.com/v4/letter/c/97f17d/32.png) [@craign30](https://discuss.elastic.co/u/craign30)\ **Post date:** [December 7, 2021, 1:17am UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/7 "2021-12-07T01:17:28Z")
No, everything is going through Logstash. There's another field, outside the grok, that gets added in the Logstash config. When the issue occurs, that field exists in Kibana, just not any fields from the grok statement. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [December 7, 2021, 1:19am UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/8 "2021-12-07T01:19:41Z")
I would start looking at conditionals and routing. Does the grok always get executed? Are you routing events beween pipelines? ---
**Author:** ![craign30](https://avatars.discourse-cdn.com/v4/letter/c/97f17d/32.png) [@craign30](https://discuss.elastic.co/u/craign30)\ **Post date:** [December 7, 2021, 1:28am UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/9 "2021-12-07T01:28:51Z")
Ok thanks, I'll do some more digging. The grok gets executed anytime "dns" is in the tags field (assigned by filebeat). I checked the bad entries in Kibana, and the dns tag is there. I do use a single filebeat instance to monitor two different log locations (dhcp and dns) on the server, but didn't think that'd be an issue. The dhcp and dns data ultimately end up in the same index in Elastic. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [December 7, 2021, 2:43am UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/10 "2021-12-07T02:43:36Z")
Do the events have a \_grokparsefailure tag in the tags field? If not then either the grok filter did not execute, or the [message] field does not exist. ---
**Author:** ![craign30](https://avatars.discourse-cdn.com/v4/letter/c/97f17d/32.png) [@craign30](https://discuss.elastic.co/u/craign30)\ **Post date:** [December 7, 2021, 2:52am UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/11 "2021-12-07T02:52:28Z")
I do see the \_grokparsefailure tag on the bad entries now. I didn't initially realize, but there was a remove\_tag entry removing it at the end of the config. ---
**Author:** ![craign30](https://avatars.discourse-cdn.com/v4/letter/c/97f17d/32.png) [@craign30](https://discuss.elastic.co/u/craign30)\ **Post date:** [December 7, 2021, 3:53am UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/12 "2021-12-07T03:53:17Z")
I had the field after the timestamp set to NUMBER, but it occasionally included a letter, so I changed it to BASE16NUM. The WORD:dns.header\_flags field is also occasionally blank, so I switched it to GREEDYDATA:dns.header\_flags. I haven't seen an unparsed entry since. Thanks for your help! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 4, 2022, 3:53am UTC](https://discuss.elastic.co/t/grok-parsing-in-logstash-for-windows-dns-debug-log/291085/13 "2022-01-04T03:53:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.