I'm writing this topic to have some help, advices, tips about how to send correctly the dns logs on a windows domain controller. I'm a beginner in ELK stack and I have to ship log of DNS role (dhcp log later).
My configuration :
- ELK Stack version 5.6.7 on a CentOS Linux release 7.4.1708
- Windows 2K12 server with dns role
I would like to send the dns log file to ELK.
I have done a test with filebeat. Output is Elasticsearch. It works but I have no filter, so the message field contains all informations and I can't use them properly.
Same test with filebeat but output is logstach. I have understood I have to parse informations but I am a little bit lost.
How I can skip the 30 first lines ?
I have found this pattern https://gist.github.com/markwalkom/3766250c5f6b6206bafcc6c23562b3fc
but I don't understand this part :
is that means the pattern works if the data is a Q, a R or a U ? But no fieds is added ?
I have done a test with packetbeat and output is elasticsearch. It works properly, I have a lot of informations. But how I can filter or skip the columns I don't want. Is it possible to do this in the yml file ?
Or the only solution is to skip the fields when I choose the logstach column ?
Is that better to use a powershell script to parse the log file, and to do an export in json or csv and analyse this export with filebeat ? If I do this, how I can resume the analyse to avoid to loose informations ? Filebeat can do this ?
Do I have other possibilites ? Is the version 6.2 offers other tools ?
If someone has already done the dns log shipping of a Windows Server, how have you done ? What solution have you choose ?
Thanks for your help.