Hello everyone. I'm going to start with saying sorry TLDR but hope someone can shine the light in the right direction. Pretty new to logstash/elastic and trying to come to grasp with the ECS field guide and want to teach myself to map fields correctly, mostly so that the elastic SIEM works or other features that are built-in to enterprise elastic, etc. works out of the box. Where i struggle is what to use for the fields as i haven't really found anything written in stone. For example in the below log (IP's redacted)
2020-12-11T11:52:36-05:00 ns3 client 134.192.2.1#59359 (pagead.l.doubleclick.net): view Internal: query: pagead.l.doubleclick.net IN AAAA +ED (134.192.1.1)
Starts simple with;
%{TIMESTAMP_ISO8601} %{WORD:???}
The second field "ns3" is the host name of the dns server. should i be mapping it as host.name, host.hostname, observer.hostname, something i make up but consistent in my environment, etc?.
The third field i can just add to my grok statement "client" and roll with the next which is the source ip of the query, or it could be considered the client.
%{TIMESTAMP_ISO8601} %{WORD:???} client %{IP:???}
Same scenario the IP could be mapped as multiple fields like; client.ip, client.address, source.address, source.ip, dns.answers.data, Local.Address, etc.
%{TIMESTAMP_ISO8601} %{WORD:} client %{IP}#%{INT:}
I dont think the integer is valuable for mapping but again how do you determine a good map for it?
%{TIMESTAMP_ISO8601} %{WORD:} client %{IP}#%{INT:} (%{NOTSPACE:???})
The next one i thought would be easy "(pagead.i.doubleclick.net)" in the dns section but wrong again. it could be dns.answers.name, dns.question.name. Im thinking this should be the dns.question though as its followed by query?
For the next three i think its straight forward;
"IN" equals dns.question.class
"AAAA" equals dns.question.type
"+ED" equals dns.header_flags
The final piece to this gem and i really appreciate you getting this far, is an IP address which is the host ip of the dns server. So should this be observer.ip, source.address, source.ip, host.ip, etc?
Maybe im over thinking this or making it more difficult than it should be?
Thanks for any feedback in advance,
Cory