Mixed JSON input and Other Log files with Multiline codec

Elastic Stack Logstash
1

I have the following configured for input. My situation that I have a mix of different file formats that I need to process.

Some files are the generic type of file, that have the same start pattern for multiline, (a date) however I am running into the problem where I have some files that are just purely json format, and seem to be getting caught up in the multiline. Am just quickly wondering if here is a workaround, or the right way to do it. Don't really want to be running different ports if I can avoid it.

Thanks in advance.

input {

beats {
            port => 4445
            codec => multiline {
                 pattern => "^\d+"
                 negate => true
                what => "previous"
            }


    }

}

2

It's pretty clear that your current multiline pattern won't work for JSON lines that begin with a curly brace. Would changing the pattern to e.g. ^(\d+|\{") help?

3

Ah yeah. Would have come to me eventually. Thanks heaps will give it a go in the morning.

Thanks again mate.

4

I was in the mode were at the input stage you would want to inspect the input type and then treat it in a particular way. I.e some logs you might want multi line and other log types you don't.

I spose you would just deal with those in the filter stage.