# DNS Queries grok pattern working on devtools but not in kibana dashboards **URL:** \ **Category:** Logstash\ **Created:** [August 26, 2023, 2:45pm UTC](https://discuss.elastic.co/t/dns-queries-grok-pattern-working-on-devtools-but-not-in-kibana-dashboards/341748 "2023-08-26T14:45:49Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Poubelle\_Dirty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/poubelle_dirty/32/122745_2.png) [@Poubelle\_Dirty](https://discuss.elastic.co/u/Poubelle_Dirty)\ **Post date:** [August 26, 2023, 2:45pm UTC](https://discuss.elastic.co/t/dns-queries-grok-pattern-working-on-devtools-but-not-in-kibana-dashboards/341748/1 "2023-08-26T14:45:49Z")
Hello, I'm new to ELK stack and I encounter a problem. I'm using the DNS grok pattern from here to parse dns queries from my bind server : [https://github.com/cjslack/grok-debugger/blob/master/public/patterns/bind](https://github.com/cjslack/grok-debugger/blob/master/public/patterns/bind) It works perfectly to parse those kind of messages : 26-Aug-2023 16:22:11.872 queries: info: client @0x7fc8302cb070 192.168.1.75#15805 ([discuss.elastic.co](http://discuss.elastic.co)): query: [discuss.elastic.co](http://discuss.elastic.co) IN AAAA +T (192.168.1.57) But in the logs, I can see that I also have those messages with HTTPS type and the native grok filter from github doesn't include this "HTTPS" Type : 26-Aug-2023 16:32:13.096 queries: info: client @0x7fc83023aff0 192.168.1.75#16181 (consent.cookiebot.eu): query: consent.cookiebot.eu IN **HTTPS** +T (192.168.1.57) So I added "HTTPS" in the grok pattern like this : BIND9\_DNSTYPE (?:A|AAAA| **HTTPS** |CAA|CDNSKEY|CDS|CERT|CNAME|CSYNC|DLV|DNAME|DNSKEY|DS|HINFO|LOC|MX|NAPTR|NS|NSEC|NSEC3|OPENPGPKEY|PTR|RRSIG|RP|SIG|SMIMEA|SOA|SRV|TSIG|TXT|URI) When trying this updated pattern in the kibana devtools with the grok debugger, it parse the logs correctly but not in my kibana dashboard, where I get a grokparsefailure each time the dns query type is "HTTPS" (other type like A and AAAA) are working perfectly. I tried to reboot my entire stack and machine but the problem is not there. Thank you ! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 26, 2023, 3:37pm UTC](https://discuss.elastic.co/t/dns-queries-grok-pattern-working-on-devtools-but-not-in-kibana-dashboards/341748/2 "2023-08-26T15:37:00Z")
If I override all the patterns in the grok filter itself ``` grok { pattern_definitions => { "BIND9_TIMESTAMP" => "%{MONTHDAY}[-]%{MONTH}[-]%{YEAR} %{TIME}" "BIND9_DNSTYPE" => "(?:A|AAAA|CAA|CDNSKEY|CDS|CERT|CNAME|CSYNC|DLV|DNAME|DNSKEY|DS|HINFO|LOC|MX|NAPTR|NS|NSEC|NSEC3|OPENPGPKEY|PTR|RRSIG|RP|SIG|SMIMEA|SOA|SRV|TSIG|TXT|URI)" "BIND9_CATEGORY" => "(?:queries)" "BIND9_QUERYLOGBASE" => "client(:? @0x(?:[0-9A-Fa-f]+))? %{IP:[client][ip]}#%{POSINT:[client][port]:int} \(%{GREEDYDATA:[bind][log][question][name]}\): query: %{GREEDYDATA:[dns][question][name]} (?<[dns][question][class]>IN) %{BIND9_DNSTYPE:[dns][question][type]}(:? %{DATA:[bind][log][question][flags]})? \(%{IP:[server][ip]}\)" "BIND9_QUERYLOG" => "%{BIND9_TIMESTAMP:timestamp} %{BIND9_CATEGORY:[bind][log][category]}: %{LOGLEVEL:[log][level]}: %{BIND9_QUERYLOGBASE}" } match => { "message" => "%{BIND9_QUERYLOG}" } } ``` Then it parses the regular log entry just fine. If I change`|HINFO|LOC|` to `|HINFO|HTTPS|LOC|` then it also parses the HTTPS log entry. So the question is where exactly are you modifying the BIND9\_DNSTYPE pattern and why do you expect the grok filter to use the modified pattern? ---
**Author:** ![Poubelle\_Dirty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/poubelle_dirty/32/122745_2.png) [@Poubelle\_Dirty](https://discuss.elastic.co/u/Poubelle_Dirty)\ **Post date:** [August 26, 2023, 3:48pm UTC](https://discuss.elastic.co/t/dns-queries-grok-pattern-working-on-devtools-but-not-in-kibana-dashboards/341748/3 "2023-08-26T15:48:01Z")
Thank you for helping. In my logstash configuration file, I use this : ```auto filter { grok { patterns_dir => ["./patterns"] match => { "message" => "%{BIND9-jij}" } } ``` And in the "patterns" dir, I have the pattern definition within a file named "bind" : ```auto BIND9_TIMESTAMP %{MONTHDAY}[-]%{MONTH}[-]%{YEAR} %{TIME} BIND9_DNSTYPE (?:A|AAAA|CAA|CDNSKEY|CDS|CERT|CNAME|CSYNC|DLV|DNAME|DNSKEY|DS|HINFO|HTTPS|LOC|MX|NAPTR|NS|NSEC|NSEC3|OPENPGPKEY|PTR|RRSIG|RP|SIG|SMIMEA|SOA|SRV|TSIG|TXT|URI) BIND9_CATEGORY (?:queries) # dns.question.class is static - only 'IN' is supported by Bind9 # bind.log.question.name is expected to be a 'duplicate' (same as the dns.question.name capture) BIND9_QUERYLOGBASE client(:? @0x(?:[0-9A-Fa-f]+))? %{IP:[client][ip]}#%{POSINT:[client][port]:int} \(%{GREEDYDATA:[bind][log][question][name]}\): query: %{GREEDYDATA:[dns][qu estion][name]} (?<[dns][question][class]>IN) %{BIND9_DNSTYPE:[dns][question][type]}(:? %{DATA:[bind][log][question][flags]})? \(%{IP:[server][ip]}\) # for query-logging category and severity are always fixed as "queries: info: " BIND9_QUERYLOG %{BIND9_TIMESTAMP:timestamp} %{BIND9_CATEGORY:[bind][log][category]}: %{LOGLEVEL:[log][level]}: %{BIND9_QUERYLOGBASE} BIND9-jij %{BIND9_QUERYLOG} ``` I Just tried the "pattern\_definitions" as you did and it works perfectly for https so it's seems that my patterns\_dir is not working It's really weird because I changed on purpose the name BIND9 by BIND9-jij to see if logstash is loading the pattern. When i spell it wrong in my logsyash conf file (for example (BIND9-jijiji), it obviously fails with an errror saying it can't load the pattern file but here when spelled correctly (BIND9-jij), there is no error so it seems everything is loaded perfectly.... ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 26, 2023, 4:09pm UTC](https://discuss.elastic.co/t/dns-queries-grok-pattern-working-on-devtools-but-not-in-kibana-dashboards/341748/4 "2023-08-26T16:09:55Z")
logstash bundles a different set of BIND9 patterns. ``` # cat /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-patterns-core-4.3.4/patterns/legacy/bind BIND9_TIMESTAMP %{MONTHDAY}[-]%{MONTH}[-]%{YEAR} %{TIME} BIND9 %{BIND9_TIMESTAMP:timestamp} queries: %{LOGLEVEL:loglevel}: client(:? @0x(?:[0-9A-Fa-f]+))? %{IP:clientip}#%{POSINT:clientport} \(%{GREEDYDATA:query}\): query: %{GREEDYDATA:query} IN %{GREEDYDATA:querytype} \(%{IP:dns}\) ``` so there is always going to be BIND9 pattern defined. Try changing your grok match to `[message]" => "%{BIND9_QUERYLOG}"` and I suspect that will be an undefined pattern. The issue may be `patterns_dir => ["./patterns"]`. That is relative to the current working directory of the logstash instance, and, particularly if you are running as a service, that may not be what you expect. Can you try an absolute path there? ---
**Author:** ![Poubelle\_Dirty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/poubelle_dirty/32/122745_2.png) [@Poubelle\_Dirty](https://discuss.elastic.co/u/Poubelle_Dirty)\ **Post date:** [August 27, 2023, 7:11am UTC](https://discuss.elastic.co/t/dns-queries-grok-pattern-working-on-devtools-but-not-in-kibana-dashboards/341748/5 "2023-08-27T07:11:37Z")
Hello, Thank you for helping, it was indeed the problem. With a full path as "pattern\_dirs", it now works. Have a nice day ! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 24, 2023, 7:12am UTC](https://discuss.elastic.co/t/dns-queries-grok-pattern-working-on-devtools-but-not-in-kibana-dashboards/341748/6 "2023-09-24T07:12:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.