Count total DNS queries

Elastic Stack Kibana
1

Hello everyone,

I'm trying to figure out the best way to count total DNS Queries per hour or day. I can easily adjust period. But what would be the best way to count this?
Packetbeat is sending data to elasticsearch and i use kibana to visualize.
So far I've created a visualization like this:

Metrics
Y-Axis Unique count of _id

Buckets
X-Axis @timestamp per hour

At first it did the trick and then started displaying errors like "10 of 21 shards failed" with " No results found".
I could also use:

Metrics
Y-Axis Count

Buckets
X-Axis @timestamp per hour

That would display me the results but would that be correct way?
I tried counting the dns.id field but I've noticed that i have a lot of duplicate DNS ID's for different client_ip addresses so that is definitely not the correct way.
I could also count the client_ip's but that way i will not have a timestamp, instead it will just display top 5 IPs with number of hits.

So how am i to reach the most accurate data? Is my second example good enough?
I have a DNS that receives really large traffic, so i cannot manually count it to make sure :slight_smile:

2

The answer might depend on which visualization you use. But a Line visualization with the X-Axis set to Date Histogram is how I might do this.

Setting Y-Axis to Count will count all documents in the index for time period. So I think that's what you want. Or you might try setting Unique Count on the dns.id field to get rid of the duplicates.

3

That is how i actually have it set. I just forgot to mention that Bucket with the X-Axis is set to Date Histogram. So that's definitely good! Nice to know, thanks!!

As for the other option, if i set it to Unique count dns.id, it will leave out duplicates, yes. But there are multiple different clients (queries from client_ip) that are using the same dns.id. So those will be ignored. Don't know why. So it will not display correct data. Just for the record, if i set it to unique count dns.id it will show approx 65k hits. If i set X Axis to Date Histogram and Y to count. It will show up as 180k hits. That seems more like accurate result.