please i need your help, i have a problem i do not know how to use the grok filter and the conditions properly .. i have a little bit 6 types of emails and i have done some filters but it does not work
if you if any of you can help me .. I will be very grateful
here is a type of email that I want to treat :
Dear Sir/Madam,
We have detected abuse from the IP address ( 197.230.107.154 ), which according to a whois lookup is on your network. We would appreciate if you would investigate and take action as appropriate. Any feedback is welcome but not mandatory.
Log lines are given below, but please ask if you require any further information.
(If you are not the correct person to contact about this please accept our apologies - your e-mail address was extracted from the whois record by an automated process. This mail was generated by Fail2Ban.)
IP of the attacker: 197.230.107.154
You can contact us by using: abuse-reply@keyweb.de
Addresses to send to:
noc_isp@meditel.ma
==================== Excerpt from log for 197.230.107.154 ====================
Note: Local timezone is +0100 (CET)
Mar 7 20:44:05 shared07 sshd[5371]: Invalid user admin from 197.230.107.154
Mar 7 20:44:05 shared07 sshd[5371]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=197.230.107.154
Mar 7 20:44:07 shared07 sshd[5371]: Failed password for invalid user admin from 197.230.107.154 port 33040 ssh2
Mar 7 20:44:08 shared07 sshd[5371]: Connection closed by 197.230.107.154 port 33040 [preauth]
there is my logstash config file :
input {
imap {
host => "imap.gmail.com"
password => "xxxxxxxxx"
user => "rouchad767@gmail.com"
port => 993
secure => true
fetch_count => 15
check_interval => 10
strip_attachments => true
folder => "Inbox"
}
}
filter {
if [from] == "abuse orange abuseorange47@gmail.com" {
grok {
match => { "message" => "%{GREEDYDATA:message}%{NOTSPACE}%{SPACE}%{GREEDYDATA:message}%{NOTSPACE}%{SPACE}%{GREEDYDATA:message}%{NOTSPACE}%{SPACE}%{GREEDYDATA:message}%{NOTSPACE}%{SPACE}%{CISCO_REASON}%{NOTSPACE}%{SPACE}%{GREEDYDATA:Attacker_IP}%{NOTSPACE}%{SPACE}%{CISCO_REASON}%{NOTSPACE}%{SPACE}%{GREEDYDATA:sender}%{NOTSPACE}%{SPACE}%{GREEDYDATA:message}%{NOTSPACE}%{SPACE}%{GREEDYDATA:receiver}%{NOTSPACE}%{SPACE}%{NOTSPACE}%{SPACE}%{GREEDYDATA:message}%{NOTSPACE}%{SPACE}%{GREEDYDATA:message}%{SPACE}%{NOTSPACE}%{SPACE}%{CISCOTIMESTAMP:time_attack}%{SPACE}%{WORD}%{SPACE}%{WORD}%{SPACE}%{NOTSPACE}%{WORD}%{SPACE}%{NOTSPACE}%{SPACE}%{GREEDYDATA:Attack}%{NOTSPACE}%{SPACE}%{CISCOTIMESTAMP:time_attack}%{SPACE}%{WORD}%{SPACE}%{WORD}%{SPACE}%{NOTSPACE}%{WORD}%{SPACE}%{NOTSPACE}%{SPACE}%{GREEDYDATA:Attack}%{NOTSPACE}%{SPACE}%{CISCOTIMESTAMP:time_attack}%{SPACE}%{WORD}%{SPACE}%{WORD}%{SPACE}%{NOTSPACE}%{WORD}%{SPACE}%{NOTSPACE}%{SPACE}%{GREEDYDATA:Attack}%{NOTSPACE}%{SPACE}%{CISCOTIMESTAMP:time_attack}%{SPACE}%{WORD}%{SPACE}%{WORD}%{SPACE}%{NOTSPACE}%{WORD}%{SPACE}%{NOTSPACE}%{SPACE}%{GREEDYDATA:Attack}" }
}
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
index => "datamail"
document_type => "email"
hosts => "localhost:9200"
}
}
please help me !!