Windows DNS Diagnostic Logs

Elastic Stack Beats
1

Hello

I have enabled DNS Debug Logging on our domain controllers and am using Filebeat to ship these logs, but the problem is the logs do not contain the ip address returned by a DNS query.

To get that level of logging, I believe I need to use DNS Diagnostic logging in Windows 2012, but the log produced by this (Microsoft-Windows-DNS-Server/Analytical) is not recognised by Winlogbeat, and is in ETL format so Filebeat is unable to read it.

Has anyone managed to import DNS Diagnostic logs into Elasticearch?

Thank you

2

Winlogbeat does not support ETW logs (Debug and Analytic), but feel free to open an enhancement request for the feature so we can track demand.

As a side note, Packetbeat has great support for DNS monitoring.

3

Thanks for the suggestion. I am trying packetbeat and it looks like a much better way to go, but I am having trouble parsing the dns.answers field.

It looks like I should be able to use a kv filter to break it up but it doesn't work:

Filter:

if "packetbeat" in [tags] {
    kv{
		value_split => ":"
	}
}

Example field:

dns.answers {
"class": "IN",
"data": "23.227.38.32",
"name": "bananas.com",
"ttl": 7199,
"type": "A"
}

Thanks

4

You shouldn't need the kv filter since the data from Packetbeat is already structured. Logstash receives the event from Packetbeat as JSON and automatically unmarshals it. What do you want to do with the data and what does your current config look like?

5

The data is all there, but I want to split up the dns.answers fields so can filter queries on "data".

The config files are very basic:

input {
	beats {
		port => 5041
		tags => ["packetbeat"]
	}

}


filter {
    if "packetbeat" in [tags] {
        kv{
		value_split => ":"
		}
    }
}

output {
        if "packetbeat" in [tags] {
                elasticsearch { hosts => ["localhost:9200"]
				index => "logstash-packetbeat-%{+YYYY.MM.dd}"
				document_type => "packetbeat"
		}
	}
}

When I look at the dns.answers field in Kibana, there is a yellow warning triangle and a message saying "objects in arrays are not well supported".

6

The tricky part about dns.answers is that it is an array of objects (see raw event below). You can run queries on it, like dns.answers.data:"66.218.75.97", but it will be difficult to create visualizations.

My sample event was indexed directly into Elasticsearch from Packetbeat and I am using the default index template provided by Packetbeat 5.x. You can route the data through Logstash as you are doing but it would be best if you used the index template provided by your Packetbeat version to ensure the mappings are correct. See the section on loading the index template and see the section about configuring Logstash for Beats to make sure you are using the template.

{
    "@timestamp": "2016-06-20T22:45:38.084Z",
    "beat": {
      "hostname": "xx",
      "name": "xx"
    },
    "bytes_in": 37,
    "bytes_out": 506,
    "client_ip": "xx",
    "client_port": 52897,
    "client_proc": "",
    "client_server": "",
    "count": 1,
    "dns": {
      "additionals_count": 5,
      "answers": [
        {
          "class": "IN",
          "data": "imap.mail.gm0.yahoodns.net.",
          "name": "imap.mail.yahoo.com.",
          "ttl": "279",
          "type": "CNAME"
        },
        {
          "class": "IN",
          "data": "67.195.236.149",
          "name": "imap.mail.gm0.yahoodns.net.",
          "ttl": "8",
          "type": "A"
        },
        {
          "class": "IN",
          "data": "66.218.75.97",
          "name": "imap.mail.gm0.yahoodns.net.",
          "ttl": "8",
          "type": "A"
        },
        {
          "class": "IN",
          "data": "98.138.74.45",
          "name": "imap.mail.gm0.yahoodns.net.",
          "ttl": "8",
          "type": "A"
        },
        {
          "class": "IN",
          "data": "98.138.74.44",
          "name": "imap.mail.gm0.yahoodns.net.",
          "ttl": "8",
          "type": "A"
        },
        {
          "class": "IN",
          "data": "98.138.74.42",
          "name": "imap.mail.gm0.yahoodns.net.",
          "ttl": "8",
          "type": "A"
        },
        {
          "class": "IN",
          "data": "67.195.236.145",
          "name": "imap.mail.gm0.yahoodns.net.",
          "ttl": "8",
          "type": "A"
        },
        {
          "class": "IN",
          "data": "67.195.236.146",
          "name": "imap.mail.gm0.yahoodns.net.",
          "ttl": "8",
          "type": "A"
        },
        {
          "class": "IN",
          "data": "66.218.74.148",
          "name": "imap.mail.gm0.yahoodns.net.",
          "ttl": "8",
          "type": "A"
        }
      ],
      "answers_count": 9,
      "authorities_count": 13,
      "flags": {
        "authentic_data": false,
        "authoritative": false,
        "checking_disabled": false,
        "recursion_available": true,
        "recursion_desired": true,
        "truncated_response": false
      },
      "id": 33912,
      "op_code": "QUERY",
      "question": {
        "class": "IN",
        "etld_plus_one": "yahoo.com.",
        "name": "imap.mail.yahoo.com.",
        "type": "A"
      },
      "response_code": "NOERROR"
    },
    "ip": "xx",
    "method": "QUERY",
    "port": 53,
    "proc": "",
    "query": "class IN, type A, imap.mail.yahoo.com.",
    "resource": "imap.mail.yahoo.com.",
    "responsetime": 39,
    "server": "",
    "status": "OK",
    "tags": [
      "xx"
    ],
    "transport": "udp",
    "type": "dns"
  }
7

dns.answers.data:"66.218.75.97"

I think as long as I can query the dns.answers array in the manner you suggested, which I didn't realise, then that will be sufficient.

Thank you for your help

8

This topic was automatically closed after 21 days. New replies are no longer allowed.