Using split filter to break multi-line messages into separate (single-line) events

Elastic Stack Logstash
1

An example message field I'm trying to split:

"337 <190>1 2019-06-27T17:05:15.711968+00:00 host app worker.1 - request_id=no_request_id organization_id=no_organization_id user_id=no_user_id hostname=no_hostname 139885407106816 INFO kafka.utils:130 log_offsets() Partition Offset: consumerGroup=consumerGroup partitionId=1 low=237919 high=237920 offset=237920 lag=0\n340 <190>1 2019-06-27T17:05:15.758428+00:00 host app worker.1 - request_id=no_request_id organization_id=no_organization_id user_id=no_user_id hostname=no_hostname 139885407106816 INFO kafka.utils:130 log_offsets() Partition Offset: consumerGroup=consumerGroup partitionId=2 low=2425733 high=2425739 offset=2425739 lag=0\n337 <190>1 2019-06-27T17:05:15.843689+00:00 host app worker.1 - request_id=no_request_id organization_id=no_organization_id user_id=no_user_id hostname=no_hostname 139885407106816 INFO kafka.utils:130 log_offsets() Partition Offset: consumerGroup=consumerGroup partitionId=3 low=260792 high=260812 offset=260812 lag=0\n337 <190>1 2019-06-27T17:05:15.916267+00:00 host app worker.1 - request_id=no_request_id organization_id=no_organization_id user_id=no_user_id hostname=no_hostname 139885407106816 INFO kafka.utils:130 log_offsets() Partition Offset: consumerGroup=consumerGroup partitionId=4 low=897807 high=897810 offset=897810 lag=0\n"

I have the following config to split this event into multiple events:

filter {
    split {
        field => "message"
        terminator => "\n"
        add_tag => ["debug:split"]
    }
}

but I still see the event isn't split, and there is no debug:split tag.

I also have other filter blocks following that one that run properly (parsing fields, removing and adding fields). Is there a problem with this split config block?

2

Unless you have config.support_escapes enabled you should use a literal newline in the configuration

terminator = "
"
2 Likes
3

Thank you!

I now see the debug:split tag and single-line events after adding to following to logstash.yml:

config.support_escapes: true
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.