Hi,
I have a Logstash input of Kafka(codec cef), that consumes arcsight CEF format events from a kafka topic and writes it to elastic with 'elasticsearch' output
I have an issue when some of the events have multiline fields.
The events stored in kafka has new line characters("\n").
for example(from kafka topic):
msg=ExecutionPolicy:Undefined\nScope:MachinePolicy\nUser:xxx\noccurrences:1
When the logstash consumes(pulls) the events from the kafka and stores them to elastic, instead of leaving it as is so Kibana could read it as a new line, it adds another backslash(escaping).
for example(json from kibana discover):
"message": "ExecutionPolicy:Undefined\\nScope:MachinePolicy\\nUser:xxx\\noccurrences:1",
example(table from kibana discover):
message
ExecutionPolicy:Undefined\nScope:MachinePolicy\nUser:xxx\noccurrences:1
I want Logstash to avoid escaping the backslash with another backslash so elk show it as new line.
good exapmple(json from kibana discover):
"message": "NLA_Enabled:Disabled\nRDS_Enabled:Disabled\nService_Startup_Type:Manual\nService_Status:Stopped\noccurrences:1",
good example(table from kibana discover):
logstash config:
input {
kafka {
bootstrap_servers=> "xxx"
codec => cef {}
topics => "xxx"
group_id => "xxx"
auto_offset_reset => "latest"
client_id => "xxx"
}
}
output {
elasticsearch {
hosts => ["xxx"]
index => "xxx"
template_name => "xxx"
user => "xxx"
password => "xxx"
ssl => true
ssl_certificate_verification => false
cacert => "xxx.cer"
}