Subsequently get only first line of multi-line grok

Elastic Stack Logstash
1

So, I'm retrieving a multi-line input just fine and storing in message. However, for throttling purposes I'd like to also just consider the first line so I'm trying to break the multi-line message up. I'm trying this:

grok {
  match => [ "message", "(?m)%{TIMESTAMP_ISO8601:timestamp} \[%{NUMBER:threadId}\] %{LOGLEVEL:level} %{DATA:logger} %{GREEDYDATA:message}" ]
  overwrite => [ "message" ]
  tag_on_failure => ["error_message_not_parsed"]
  break_on_match => false
}

...

# create a copy of the message
mutate {
	add_field => { "message_array" => "%{message}" }
}

# make new copy become an array whereby each line is an element
mutate {
	split => { "message_array" => "\n" }
}

# keep first line only
mutate {
	replace => { "message_array" => "%{[message_array][0]}" }
}	

# let's throttle events after first occurrence each 5 minutes for email purposes
throttle {
	before_count => -1
	after_count => 1
	period => 300
	key => "%{city}%{message_array}"
	add_tag => "throttled"
}

message_array is still the same as message however ( not just the first line ), so appears I'm doing something wrong with the split?

1 Like
2

It was the mutate split on "\n" that does not work - worked around using:

ruby {
   code => "event['message_array'] = event['message_array'].split(/\n/)"
}
1 Like