Hello,
I'm trying to get the first line of the message field because this is an important information to keep.
The [message] line, when I set the output to file using rudydebug option looks like this:
"message" => "An HTTP request was received. See audit 510 with the same Instance ID for headers. \n\nInstance ID: 8ac2ca87-8958-4e0e-a79a-8be672341b5b \n\nActivity ID: 62f3ac59-1f9b-9254-97c8-b00a8bd00357 \n\nRequest Details: \n Date And Time: 2021-11-09 14:15:16 \n Client IP: 111.111.1.111 \n HTTP Method: POST \n Url Absolute Path: /adfs/oauth2/token/ \n Query string: - \n Local Port: 443 \n Local IP: 100.132.4.89 \n User Agent: Windows-AzureAD-Authentication-Provider/1.0 \n Content Length: 4223 \n Caller Identity: - \n Certificate Identity (if any): - \n Targeted relying party: - \n Through proxy: False \n Proxy DNS name: -"
I would like to have this:
"An HTTP request was received. See audit 510 with the same Instance ID for headers."
Which is the first line before a \n.
In logstash I tried several solution I've found on this forum, but there is no one working:
mutate {
copy => { "message" => "message_head" }
}
mutate {
gsub => [ "message_head", "^([^\n]*)$", "\1" ]
}
I also tried this:
mutate {
copy => { "message" => "message_head" }
}
mutate {
split => ["message_head", "
"]}
mutate {
replace => { "message_head" => "%{message_head[0]}" }
}
And:
mutate { split => [ "message_head" => "\n" ] }
The result is almost all the same; the 'split' or 'gsub' are just not being applied:
"message_head" => "An HTTP request was received. See audit 510 with the same Instance ID for headers. \n\nInstance ID: 8ac2ca87-8958-4e0e-a79a-8be672341b5b \n\nActivity ID: 62f3ac59-1f9b-9254-97c8-b00a8bd00357 \n\nRequest Details: \n Date And Time: 2021-11-09 14:15:16 \n Client IP: 111.111.1.111 \n HTTP Method: POST \n Url Absolute Path: /adfs/oauth2/token/ \n Query string: - \n Local Port: 443 \n Local IP: 100.132.4.89 \n User Agent: Windows-AzureAD-Authentication-Provider/1.0 \n Content Length: 4223 \n Caller Identity: - \n Certificate Identity (if any): - \n Targeted relying party: - \n Through proxy: False \n Proxy DNS name: -",
"message" => "An HTTP request was received. See audit 510 with the same Instance ID for headers. \n\nInstance ID: 8ac2ca87-8958-4e0e-a79a-8be672341b5b \n\nActivity ID:62f3ac59-1f9b-9254-97c8-b00a8bd00357 \n\nRequest Details: \n Date And Time: 2021-11-09 14:15:16 \n Client IP: 111.111.1.111 \n HTTP Method: POST \n Url Absolute Path: /adfs/oauth2/token/ \n Query string: - \n Local Port: 443 \n Local IP: 100.132.4.89 \n User Agent: Windows-AzureAD-Authentication-Provider/1.0 \n Content Length: 4223 \n Caller Identity: - \n Certificate Identity (if any): - \n Targeted relying party: - \n Through proxy: False \n Proxy DNS name: -",
In some solution I've found, people said it was working, such as this thread: https://discuss.elastic.co/t/add-winlogbeat-option-to-truncate-security-message-field-to-just-first-line/49409, but it is not the case on my side.
To see when I see into Kibana, please look at the attached screenshot.
You will notice the current filter, which is:
mutate {
copy => { "message" => "message_head" }
}
mutate {
gsub => [ "message_head", "^([^\n]*)$", "\1" ]
}
... did nothing.
If I test my regular expression into a "regular expression tester", I can see that it is working:
So I'm out of solutions here...
Is there anybody who can help me out with this issue?
Thank you and best regards,
Yanick