Hello,
I'm using KV filter in Logstash. Logstash configuration:
filter {
#PARSING LOG
#------------------------------------------
grok {
patterns_dir => ["/etc/logstash/grok-patterns"]
break_on_match => true
match => { "message" => [ "%{CYBERARC_LOG}" ] }
overwrite => [ "message" ]
tag_on_failure => ["not_parsed", "not_parsed_cyberarc"]
}
#MAPPING DATE
#------------------------------------------
if [timestamp] {
#https://discuss.elastic.co/t/logstash-date-parse-failure-for-jdbc-input/92713/2
#https://github.com/logstash-plugins/logstash-filter-date/issues/95
mutate {
convert => { "timestamp" => "string" }
}
date {
locale => "en"
match => ["timestamp", "ISO8601", "yyyy-MM-dd'T'HH:mm:ss" ]
target => "@timestamp"
remove_field => ["timestamp"]
}
}
# KV FILTER
#------------------------------------------
if [cef_message] {
kv {
source => "cef_message"
transform_key => "lowercase"
trim_key => "no"
trim_value => "no"
field_split => " "
whitespace => "strict"
prefix => "kv_"
#remove_field => ["cef_message"]
}
}
}
There is a log:
<5>1 2021-08-24T09:13:26Z CDIS000PHANT642 CEF:0|Cyber-Ark|Vault|12.2.0000|99|Open File|5|act=\"Open File\" fname=\"Root\\Policies\\Policy-CyberArkPTA.ini\" cs1Label=\"Affected User Name\" cs1=\"test\" cs2Label=\"Safe Name\" cs2=\"PasswordManagerShared\" cs3Label=\"Device Type\" cs3=\"test\" cs4Label=\"Other information message\" cs4=\"information\" cs5Label=\"Other information\" cs5=\"test\" cn1Label=\"Request Id\" cn1=\"test\" cn2Label=\"Ticket Id\" cn2=\"test\"
From message was parsed a field with name cef_message.
"cef_message": "act=\"Open File\" fname=\"Root\\Policies\\Policy-CyberArkPTA.ini\" cs1Label=\"Affected User Name\" cs1=\"test\" cs2Label=\"Safe Name\" cs2=\"PasswordManagerShared\" cs3Label=\"Device Type\" cs3=\"test\" cs4Label=\"Other information message\" cs4=\"information\" cs5Label=\"Other information\" cs5=\"test\" cn1Label=\"Request Id\" cn1=\"test\" cn2Label=\"Ticket Id\" cn2=\"test\" "
KV filter was applied on cef_field field.
Very strange behaving is that information is trimmed to informati.
"kv_act": "Open File"
"kv_cn1": "test",
"kv_cn1label": "Request Id",
"kv_cn2": "test",
"kv_cn2label": "Ticket Id",
"kv_cs1": "test",
"kv_cs1label": "Affected User Name",
"kv_cs2": "PasswordManagerShared",
"kv_cs2label": "Safe Name",
"kv_cs3": "test",
"kv_cs3label": "Device Type",
"kv_cs4": "informati",
"kv_cs4label": "Other information message",
"kv_cs5": "test",
"kv_cs5label": "Other informati",
"kv_fname": "Root\\Policies\\Policy-CyberArkPTA.ini",
Look at fields:
"kv_cs5label": "Other informati"
"kv_cs4label": "Other information message"
"kv_cs4": "informati"
If the text ends with "on", it removes the text "on".
I'm using Logstash 7.8.1. Can anyone help me to resolve this problem?