Logstash config - Kafka and CEF

Hi,

I have a Logstash input of Kafka(codec cef), that consumes arcsight CEF format events from a kafka topic and writes it to elastic with 'elasticsearch' output

I have an issue when some of the events have multiline fields.
The events stored in kafka has new line characters("\n").
for example(from kafka topic):

msg=ExecutionPolicy:Undefined\nScope:MachinePolicy\nUser:xxx\noccurrences:1

When the logstash consumes(pulls) the events from the kafka and stores them to elastic, instead of leaving it as is so Kibana could read it as a new line, it adds another backslash(escaping).
for example(json from kibana discover):

"message": "ExecutionPolicy:Undefined\\nScope:MachinePolicy\\nUser:xxx\\noccurrences:1",

example(table from kibana discover):

message
ExecutionPolicy:Undefined\nScope:MachinePolicy\nUser:xxx\noccurrences:1

I want Logstash to avoid escaping the backslash with another backslash so elk show it as new line.
good exapmple(json from kibana discover):

"message": "NLA_Enabled:Disabled\nRDS_Enabled:Disabled\nService_Startup_Type:Manual\nService_Status:Stopped\noccurrences:1",

good example(table from kibana discover):
image

logstash config:

input {
 kafka {
	    bootstrap_servers=> "xxx"
		codec => cef {}
		topics => "xxx"
		group_id => "xxx"      
		auto_offset_reset => "latest"
		client_id => "xxx"
     }
}
output {  
	elasticsearch {
		hosts => ["xxx"]		
		index => "xxx"
		template_name => "xxx"
		user => "xxx"
		password => "xxx"
		ssl => true
        ssl_certificate_verification => false
        cacert => "xxx.cer"
	}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.