Hi Folks,
it has been some time since i asked my last question here and i learned a lot in the meantime, but i still feel like a bloody beginner from time to time.
Here comes my Situation:
I am filtering several logs from different sources and i am trying to deploy some kind of alerting for some of the events via mail.
The Pipeline is loading, Events are passing, i even testet the mail-component (and sent around 15k mails to me by accident XD), so those parts are defenitely working as designed. The other filters (look below if unpatient) are working fine aswell. The only thing that does not seem to be working is the part of:
if (([fields.loglevel] == "error") and ([fields.env] == "ekp")) {
mutate {
add_field => { "shouldimailit" => "You should have Mailed me - finally!" }
add_tag => ["shouldmail"]
}
}
For example one of the events after passing logstash (note that there is no tag):
@timestamp|2020-02-12 @ 15:39:08.074|
@version|1|
_id|40jWOXABPVew1GGU5QBC|
_index|filebeat-7.4.2-2020.02.12|
_score| - |
_type|_doc|
agent.ephemeral_id|de1e2668-6d03-45b4-bd5f-f440691c2562|
agent.hostname|dehze01-lsv841.sec.rhs.zz|
agent.id|aa2e58b6-128b-4d55-85b1-b86f8aafdc69|
agent.type|filebeat|
agent.version|7.4.2|
ecs.version|1.1.0|
fields.env|ekp|
fields.loggername|[org.keycloak.adapters.RefreshableKeycloakSecurityContext]|
fields.loglevel|error|
fields.message|Refresh token failure status: 400 {"error":"invalid_grant","error_description":"Refresh token expired"}|
fields.stage|prod|
fields.thread|default task-77|
host.name|dehze01-lsv841.sec.rhs.zz|
input.type|log|
log.file.path|/software/bnp/wildfly_ekp/standalone/log/server.log|
log.offset|420 289 111|
message|2020-02-12 15:39:08,074 ERROR [org.keycloak.adapters.RefreshableKeycloakSecurityContext] (default task-77) Refresh token failure status: 400 {"error":"invalid_grant","error_description":"Refresh token expired"}|
tags|beats_input_codec_plain_applied|
Whole filter:
filter {
grok {
match => {
"message" => "\A%{TIMESTAMP_ISO8601:fields.timestamp}%{SPACE}%{LOGLEVEL:fields.loglevel}%{SPACE}%{NOTSPACE:fields.loggername}%{SPACE}\(%{DATA:fields.thread}\)%{SPACE}%{GREEDYDATA:fields.message}"
}
match => {
"message" => "\A%{TIMESTAMP_ISO8601:fields.timestamp}%{SPACE}%{LOGLEVEL:fields.loglevel}%{SPACE}%{NOTSPACE:fields.loggername}%{SPACE}%{GREEDYDATA:fields.message}"
}
#match => { "message" => "%{COMBINEDAPACHELOG}" }
}
date {
match => ["fields.timestamp", "ISO8601"]
target => "@timestamp"
}
mutate {
remove_field => ["fields.timestamp"]
}
# hier wird klein geschrieben!
mutate {
lowercase => [ "fields.env" ]
lowercase => [ "fields.loglevel" ]
lowercase => [ "fields.stage" ]
}
# Filter für Keycloak von ca 01.12.2019
if (([fields.loglevel] == "error") and ([fields.message] =~ "^Error from SAP:(.*)")) {
mutate {
replace => { "fields.loglevel" => "warn" }
add_tag => ["reshaped"]
}
}
# Filter für EKP lt. Domme vom 14.01.2020
if (([fields.loglevel] == "error") and ([fields.env] == "ekp") and ([fields.message] =~ "^UT005022: Exception generating error page \/pages\/error\.jsf: javax\.servlet\.ServletException: Index: 0, Size: 0(.*)")) {
mutate {
replace => { "fields.loglevel" => "warn" }
add_tag => ["reshaped"]
}
}
# Filter für EKP lt. Domme vom 16.01.2020
if (([fields.loglevel] == "error") and ([fields.env] == "ekp") and ([fields.message] =~ "^UT000010: Session is invalid(.*)")) {
mutate {
replace => { "fields.loglevel" => "warn" }
add_tag => ["reshaped"]
}
}
# Alle (restlichen) EKP-Error auf shouldmail setzen
if (([fields.loglevel] == "error") and ([fields.env] == "ekp")) {
mutate {
add_field => { "shouldtagit" => "You should have tagged me!" }
add_tag => ["shouldmail"]
}
}
}
The big question: What am I missing? Why am I not getting Emails for this event?