Yes, the example query/aggregation I posted above requires Packetbeat 5.0 because it uses the dns.question.etld_plus_one field. Using Packetbeat 5.0 will simplify the overall setup because it doesn't require Logstash or any of the extra scripts used in blog post and example. You just capture the traffic to Elasticsearch and execute the query I provided. The query provides you with a list of domains that have a high number of unique hostnames. It also provides the total number of bytes used in requests and responses to/from these domains. You can adjust the unique_domains threshold and you can modify the whitelist.
Here's an example response from the query:
{
"took": 1271,
"timed_out": false,
"_shards": {
"total": 63,
"successful": 63,
"failed": 0
},
"hits": {
"total": 34829,
"max_score": 0,
"hits": []
},
"aggregations": {
"by_domain": {
"doc_count_error_upper_bound": 37,
"sum_other_doc_count": 3231,
"buckets": [
{
"key": "sophosxl.net.",
"doc_count": 2971,
"unique_hostnames": {
"value": 1005
},
"total_bytes_in": {
"value": 154718
},
"total_bytes_out": {
"value": 656802
}
},
{
"key": "syncthing.net.",
"doc_count": 741,
"unique_hostnames": {
"value": 10
},
"total_bytes_in": {
"value": 33973
},
"total_bytes_out": {
"value": 137054
}
}
]
}
}
}