Multiline codec in input

Elastic Stack Logstash
1

I have a scenario where filebeat send information from access.log as well as application.log. The application log can be multiline.

so in the logstash.conf i have a multiline codec looking for a timestamp as the pattern.

input {

beats {
 type => beats
 port => 5044
     codec => multiline{
      pattern => "(^\<%{MONTHDAY}\s%{MONTH}\s%{YEAR}\s%{HOUR}:%{MINUTE}:%{SECOND}\>)"
      negate => true
      what => "previous"
     }

}
}

but the access.log starts with IP address when I ingest access.log data to logstash it treats all the lines into one message

961×374 15.7 KB

what is the best solution for this kind of situation

I see couple of pointers

or use

filebeat multiline, I don't want to do this approach because I have lot of harvesters on application log files (20 micro services deployed on a single container)

thanks
rakesh

2

This is the best option.

You don't have automation to handle it?

3

Mark,

Ended up using multiline filebeat.

Automation is there, My only concern was how much additional cpu and memory will be consumed by filebeat, if I put the multiline logic onto the client machine.