It works fine, but I have a lot of information I don't want, like www.google.com, www.facebook.com, so I want to remove those lines.
I already exclude some domain with yml configuration file on my servers but I want to centralize.
What's is the best way with logstash ? multiline, drop or another filter ?
And is it possible to place all domains I don't want in a text file and the filter use this text file to remove the lines ?
On my windows server, I have a log file with a lot of line like this :
22/03/2019 13:55:30 0E28 PACKET 0000007F67E71C00 UDP Rcv 172.20.18.26 92d0 Q [0001 D NOERROR] A (3)www(8)facebook(3)com(0)
22/03/2019 13:55:30 0E28 PACKET 0000007F67E71C00 UDP Rcv 172.20.18.26 92d0 Q [0001 D NOERROR] A (3)www(8)facebook(3)com(0)
22/03/2019 13:56:32 0E28 PACKET 0000007F628781B0 UDP Rcv 172.20.18.38 3b1a Q [0001 D NOERROR] A (3)www(4)bing(3)com(0)
all the lines are sent to logstash.
The filters I used is in my first message.
I use grok to parse the line and add some fields.
I want to drop the lines containg" (3)www(4)bing(3)com(0)" so I have done this configuration :
filter {
if "windows_dnslog" in [tags]
{
grok {
match => { "message" => "%{DATESTAMP:dns_time} %{GREEDYDATA:dns_thread}%{SPACE}% {WORD:dns_context} %{GREEDYDATA:dns_id} %{GREEDYDATA:dns_protocol} %{GREEDYDATA:dns_sr_indicator} %{IP:dns_remote_ip}%{SPACE}%{WORD:dns_xid} ?(%{WORD:dns_response}|%{SPACE:dns_response})%{SPACE}%{DATA:dns_opcode} \[%{GREEDYDATA:dns_responsecode}\] %{WORD:dns_questiontype}%{SPACE}%{GREEDYDATA:dns_question_name}" }
}
if [dns_question_name] == "(3)www(4)bing(3)com(0)" {
drop {}
}
}
}
I dont' want to repeat X drop filter for X domains I don't want, so it's possible to put all domains in a text file and logstash uses a kind of loop to know the domain to be drop ?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.