SIEM can't detect DNS activity to Internet

Dear, I have running an ELK server 7.7 with Elastic SIEM. I have enabled the related pre-built nmap and tcpdump signals detection rules and I could detect these type of "attacks".

But now I have enabled the "DNS Activity to Internet" signal detection rule, it looks for events in the filebeat-* index pattern by default, but after try to connect from some hosts to Internet using 8.8.8.8 Google DNS, the SIEM can't detect this dangerous traffic.

After that I duplicated the signal detection rule and I add the packetbeat-* index pattern in order to search the DNS events, but I still can't detect nothing at all.

Please can you tell me how to config filebeat or packetbeat in order to detect DNS traffic to Internet? I tell you again that the pre-built "DNS Activity to Internet" signal detection rule look into filebeat-* index pattern.

Thanks a lot in advance.

Hi Robert - so the "DNS Activity to the Internet Rule" should work with most any network data using the Elastic Common Schema (ECS.) This schema defines common fields like source.ip, destination.port and destination.ip which are used by this rule. The search used by the rule is this:

destination.port:53 and source.ip:(10.0.0.0/8 or 172.16.0.0/12 or 192.168.0.0/16) and not destination.ip:(10.0.0.0/8 or 127.0.0.0/8 or 169.254.169.254/32 or 172.16.0.0/12 or 192.168.0.0/16 or 224.0.0.251 or 224.0.0.252 or 255.255.255.255 or "::1" or "ff02::fb")

Let's troubleshoot this. If you manually run this search against your Packetbeat index, you should see events returned for the DNS traffic you described. If no events are returned, I can think of a few possible reasons.

  1. The Packetbeat instance monitoring this traffic is older and not using ECS fields? I tested this search just now in Packetbeat 7.6.1. Most recent versions of Packetbeat should generate data compatible with this rule.

  2. The Packetbeat instance has a vantage point that sees post-NAT traffic and sees the packets sourcing from a routable / public IP address instead of a private IP. Try searching for "destination.ip: 8.8.8.8 and destination.port: 53" and see if that finds the events coming from a different source IP.

  3. The Packetbeat instance did not record the packets because they did not traverse a network interface, mirror, span or tap monitored by Packetbeat?

  4. The Packetbeat instance did not record the packets because they were dropped by a network interface, mirror, span or tap monitored by Packetbeat?

  5. Could the DNS packets have been dropped by an intermediate firewall or router ACL before they traversed a network segment monitored by Packetbeat? Were the DNS queries successful?

Dear Craig, thanks for your response.

I have an ELK server 7.7.0 and I have two Linux boxes with packetbeat and filebeat 7.7.0.

Server and beat boxes are in the same network segment, no NAT and no any other thing in the middle. In the beat box I try to query 4.4.4.4 Google DNS, I run tcpdump and I can see packets going to 4.4.4.4:53. I have not response from 4.4.4.4 because it is blocket by firewall, but the Linux box with packetbeat tries to get it so I suppose the DNS Internet activity detection rule is fired up.

When I run the below query against the packetbeat index:

GET /packetbeat-7.7.0/_search
{
"query": { "match_all": {} },
"sort": [
{ "destination.ip": "asc" }
]
}

I obtain and see that ip: "4.4.4.4" (Google DNS) and not destination.ip, so I'm confused....can you help again please? Regards !!!

"hits" : [
{
"_index" : "packetbeat-7.7.0-2020.05.14-000001",
"_type" : "_doc",
"_id" : "QizsnnIBuVNl7NxaIPft",
"_score" : null,
"_source" : {
"@timestamp" : "2020-06-10T15:49:48.959Z",
"server" : {
"ip" : "4.4.4.4",
"port" : 53
},
"query" : "class IN, type A, www.sintys.gob.ar",
"source" : {
"ip" : "172.31.28.7",
"port" : 46490,
"bytes" : 35
},
"status" : "Error",
"event" : {
"start" : "2020-06-10T15:49:48.959Z",
"kind" : "event",
"category" : "network_traffic",
"dataset" : "dns"
},
"type" : "dns",
"client" : {
"ip" : "172.31.28.7",
"port" : 46490,
"bytes" : 35
},
"resource" : "www.sintys.gob.ar",
"dns" : {
"op_code" : "QUERY",
"header_flags" : [
"RD"
],
"type" : "query",
"flags" : {
"recursion_desired" : true,
"recursion_available" : false,
"authentic_data" : false,
"checking_disabled" : false,
"authoritative" : false,
"truncated_response" : false
},
"response_code" : "NOERROR",
"id" : 53781,
"additionals_count" : 0,
"question" : {
"class" : "IN",
"etld_plus_one" : "sintys.gob.ar",
"registered_domain" : "sintys.gob.ar",
"top_level_domain" : "gob.ar",
"subdomain" : "www",
"name" : "www.sintys.gob.ar",
"type" : "A"
},
"authorities_count" : 0,
"answers_count" : 0
},
"host" : {
"containerized" : false,
"ip" : [
"172.31.28.7"
],
"mac" : [
"00:50:56:b2:97:b1"
],
"hostname" : "MITLTOCS-Agent02",
"architecture" : "x86_64",
"name" : "MITLTOCS-Agent02",
"os" : {
"name" : "Debian GNU/Linux",
"kernel" : "4.9.0-3-amd64",
"codename" : "stretch",
"platform" : "debian",
"version" : "9 (stretch)",
"family" : "debian"
},
"id" : "1436e041f70b4887887077ee84a0bae1"
},
"network" : {
"direction" : "outbound",
"community_id" : "1:PpSYmtSvRXwFbj45QauJiIpR63E=",
"bytes" : 35,
"type" : "ipv4",
"transport" : "udp",
"protocol" : "dns"
},
"agent" : {
"version" : "7.7.0",
"type" : "packetbeat",
"ephemeral_id" : "bfa97b80-78d2-4810-9b30-9ab964af614f",
"hostname" : "MITLTOCS-Agent02",
"id" : "649569ab-b23d-487d-af4b-50654252e501"
},
"ecs" : {
"version" : "1.5.0"
},
"error" : {
"message" : "No response to this query was received"
},
"method" : "QUERY",
"destination" : {
"port" : 53,
"ip" : "4.4.4.4"
}
},
"sort" : [
"4.4.4.4"
]
},

If you search the packetbeat-* index in Discover, do you see results like this?

Time	source.ip		destination.ip	destination.port
		10.10.10.10		4.4.4.4			53

Dear Craig, effectively in Discover --> Packetbeat I can see an output like you told me:

Jun 10, 2020 @ 13:25:00.000 172.31.28.6 8.8.8.8 53

Jun 10, 2020 @ 13:24:50.000 172.31.28.6 8.8.8.8 53

Jun 10, 2020 @ 13:24:40.000 172.31.28.6 8.8.8.8 53

I tell you again the server 172.31.28.6 is trying to resolve public hostnames against 8.8.8.8, but 8.8.8.8 is blocked by the corporate firewall, so there is no incoming DNS traffic.

And the DNS Internet Activity signal detection rule searches into filebeat-* by default, and I've added packetbeat-*.

I'd appreciate your help please.

Regards !!!

Can you share a screenshot of the modified rule?

Yes, of course Craig, here it is.

Thanks a lot again!!!

OK that looks good of course. Is there anything in "Failure History"?

I see no "last signal" status so I assume the Signal Count and Signals sections further down in the rule are empty. Can you confirm there are no signals for this rule by searching for *DNS* in the Detections page?

Back in the custom rules list, if you click "Monitoring" can you share the values for this rule? Are there any errors or gaps in the monitoring data or stats for this rule?

One other thing occurred to me - in your Packetbeat search results, can you confirm that those fields are named like this and not something else -

source.ip                   destination.ip           destination.port
172.31.28.6               8.8.8.8                        53

Hi Craig, I tell you:

  1. In the Packetbeat results, the fields are named as I've shown to you:

source.ip / destination.ip / destination.port

  1. In "Failure History", there is nothing at all, it's empty.

  2. I see no "last signal" status and the Signal Count and Signals sections further down in the rule are empty. I confirm there are no signals for this rule by searching for *DNS* in the Detections page.

  3. The rule monitoring:

Please, If there is something else I'd can check, tell me what.

Greetings !!!

OK let’s rule out timestamps. Which field are you using for the timestamp in your packetbeat index?

Please try these steps;

  1. Modify your cloned DNS rule to lookback five hours instead of five minutes.
  2. Generate the DNS traffic and verify the events are found in Discover.
  3. Search for DNS signals to see if the rule fired on the events.

If no results, please repeat with a longer lookback if possible?

Craig,

In my custom DNS Activity rule I've modified the look-back time to 5 hours:

image.png

After that I've generated the DNS traffic to 8.8.8.8 Google DNS and I can found the events in Discover looking at Packetbeat index:

Finally I went to Detection --> Signals but no DNS rule was fired at all.

After that I've repeated with a look-back time of 10 hours, but no DNS rule was fired again.

The packetbeats from my test servers send the output to Elasticsearch, not to Logstash.

Is there any chance to view a related log file for a deeper analysis ???

Thanks a lot again !!!

OK looking at your screenshots I think you may be in the EST time zone? Let's check further for timestamp issues. Can you do the following:

  1. check the system time and time zone on your endpoint

  2. check the same on the Packetbeat host

  3. Generate the DNS traffic again and note the system time on the host where you ran this.

  4. Search for the Packetbeat events again for this DNS traffic and gather the values of the following fields from one or more Packetbeat events:

     event.start
     event.end
     @timestamp
    
  5. Do the timestamps match the system time you noted in step 3?

Dear Craig, I tell you:

ELK server --> Attached to NTP server / GMT-3
Packetbeat host --> Attached to the same NTP server / GMT-3

I executed the "date" command on both hosts and the date and time are the same.

I generated DNS traffic at Packetbeat host:

host www.google.com 8.8.8.8

These are Packetbeats events on Discovery:

I reviewed the Timezone for Date formatting:

And the DNS Internet Activity rule is not fired at all...

I could put to work the Nmap, Tcpdump and Whoami rules....but couldn't this DNS and the SMTP to Internet rules. In both DNS and SMTP to Internet rules, the default index is filebeat-*, but with just this index they don't work because some fields like destination.port don't appear in filebeat index.

Any idea?

Thanks a lot again !!!

I'm guessing you are running the other rules (nmap, tcpdump, whoami) on auditbeat data? I see the auditbeat index in the rule details - does the search also return events from the auditbeat index for the dns connections to 8.8.8.8, if you connect from an auditbeat instance? Does the rule create a signal on this activity from an auditbeat instance?

one more thing - it's not possible the dates in the Packetbeat events were shifted into the future is it? Did those timestamps agree with the local system time when you generated the DNS activity?

There is one issue where timestamps can be handled incorrectly which is covered here: https://github.com/elastic/kibana/issues/57649

As a workaround, if you check and update your field mapping/beats config/ingest pipeline to ensure @timestamp is being ingested as ISO 8601 with timezone designators (e.g. 2020-02-11T23:59:55Z ) that may be the fix.

Hi Craig, I tell you now:

  1. Auditbeat index does not have the DNS events, it doesn't work.

  2. In the duplicate DNS rule, I deleted filebeat-* and auditbeat-* patterns and I just defined packetbeat-*, and in this case it works OK !!!

  3. In the signal detection window the @timestamp for a given signal is the rule running time, and it doesn't match the time.event_start and time.event_end from the packetbeat-* pattern....I think this is OK, isn't it ???

  4. Now I think I can't use a rule with more than one index to search into, this is my problem and I don't know the reason.

  5. I couldn't understand the time definition problem you pointed to me. Maybe I have to read in depth, but my question is this: is there any easy way to check if my Date/Time format is OK ???

Special thanks again !!!

Dear Craig, how are you?

I want to tell you that after ELK upgrade from 7.7.0 to 7.7.1, the SNS Activity to Internet works OK inspecting two indexes. Maybe it was a bug, resolved in this new stable version.

Thanks a lot, cheers !

Hi Robert - glad to hear that. I tested this myself in 7.7.1 and it works ok here too. If you should receive hits from this rule, you can also use Auditbeat or Winlogbeat data to identify and investigate the process generating such DNS traffic, if you wish. Regards, Craig

Dear Craig, just a last question please:

When you say:

"If you should receive hits from this rule, you can also use Auditbeat or Winlogbeat data to identify and investigate the process generating such DNS traffic"

Does this imply that I have to investigate the process generating the DNS traffic in Discover option from Kibana, selecting and searching into auditbeat-* and filebeat-*? Or is there any specific way to do this analysis? Because now I need to know if the DNS query to the Internet was made by host command, dig command, etc.

Thanks a lot again !!!