Trouble with \n in dissect filter mapping



I am using ELK GA 6.3.0. I use Logstash to read messages from Kafka as JSON. I have a message like;

<Mar 23, 2018 7:40:04:108 AM> <test_entry_1>\n <SEVERE: logline1\nlogline2\n>

I am using the dissect filter like;

dissect {
	mapping => {
		"message" => "<%{timestamp}> <%{entry1}>%{entry2}"

This works fine. But it will make entry2 like \n <SEVERE: logline1\nlogline2\n>. I would like to remove \n and < > from entry2, so that I changed the filter like;

dissect {
	mapping => {
		"message" => "<%{timestamp}> <%{entry1}>\n <%{entry2}\n>"

But now, the result is incorrect, means, there will be < and > in entry1 and the message is not getting dissected properly. Why is this happening and how to fix this?



I do not understand dissect well enough to say why it happens, but you could move forward using

    mutate { gsub => [ "entry2", "[><]", "", "entry2", "
", " "  ] }



Cool, but that looks like my CPU usage will increase because of an additional filter :wink:

(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.