Logstash file input/output with multiline codec

Elastic Stack Logstash
1

Hello, i'm new in logstash and i'm encountering some troubles on building a config that works properly with my needings.
What i want to do is:
-Input files from multiples folders (application logs, so with java exception)
-Use the multiline codec for java exception
-Output files in multiple folder, and i want to output only the raw log, with the multiline modify but without any type of header or additional information

The issue that i'm encountering is that if i use the multiline codec in the input, it works, but i'll have the json_line as default output codec. If i specify a different codec in output, the multiline doesn't work properly.

Instead, if i use the multiline codec in output, it won't work because during the input it will add the logstash header, that cause the multiline malfunction.
Can you help me to solve this? Thanks in advance.

Here my config (now i'm trying with only one folder as input/output):

    input {
        file {
                path => "/folder/*.log"
                tags => "host1"
    				codec => multiline {
    					pattern => "^(%{MONTHDAY}-%{MONTH}-%{YEAR} %{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE})|(%{TIMESTAMP_ISO8601}%{SPACE})"
    					negate => true
    					what => "previous"
    				}
    		}
    }

    filter {
    }

    output {
            if "host1" in [tags]{
                    file {
                            path => "/folder/out.txt"
    						}
    				}
            }
2

A multiline codec on an output does not modify the output.

What do you mean by that?

3

for example if i use
codec => plain
in the output, the multiline seems working in the other way.
As an example, with the multiline i need to aggregate in a single line the java exception and i'm trying to match lines that are not beginning with a date. Now he's aggregating lines starting with date to the ones that doesn't have it.
Example:


at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)2021-03-17T17:45:39.201Z fccapsiem02 2021-01-22 10:09:54.656  INFO 2412396 --- [ost-startStop-1] i.i.o.o.ospzWebBack.OspzWebBackAppMain   : No active profile set, falling back to default profiles: default2021-03-17T17:45:39.211Z fccapsiem02 2021-02-17 10:07:29.387  INFO 3346302 --- [nio-8080-exec-4] o.s.web.servlet.DispatcherServlet        : Completed initialization in 21 ms2021-03-17T17:45:39.194Z fccapsiem02 22-Jan-2021 10:09:53.538 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.

He's doing the opposite :smiley:

4

If your input looks like

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
2021-03-17T17:45:39.201Z fccapsiem02
2021-01-22 10:09:54.656  INFO 2412396 --- [ost-startStop-1] i.i.o.o.ospzWebBack.OspzWebBackAppMain   : No active profile set, falling back to default profiles: default
2021-03-17T17:45:39.211Z fccapsiem02
2021-02-17 10:07:29.387  INFO 3346302 --- [nio-8080-exec-4] o.s.web.servlet.DispatcherServlet        : Completed initialization in 21 ms
2021-03-17T17:45:39.194Z fccapsiem02
22-Jan-2021 10:09:53.538 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.

and your output looks like

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)\n2021-03-17T17:45:39.201Z fccapsiem02 \n2021-01-22 10:09:54.656  INFO 2412396 --- [ost-startStop-1] i.i.o.o.ospzWebBack.OspzWebBackAppMain   : No active profile set, falling back to default profiles: default\n2021-03-17T17:45:39.211Z fccapsiem02 \n2021-02-17 10:07:29.387  INFO 3346302 --- [nio-8080-exec-4] o.s.web.servlet.DispatcherServlet        : Completed initialization in 21 ms\n2021-03-17T17:45:39.194Z fccapsiem02\n22-Jan-2021 10:09:53.538 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.

that would certainly indicate that your configuration has

negate => false

(which is the default).

5

exactly, the problem is that in my config (as you can see in the first post) i have it set to true....
Could it be that, for some reason, it is misinterpreting it?

7

i just tried to set i to false (it was true) and now is working as i want it to work, which is the opposite of what is expected from this config.....strange :smiley:
The problem is that logstash is processing logs in a random order...i have, for example, the continuation of one exception that is aggregated at the beginning of another.
And also for the lines that are matched with multiline (lines that are not starting with the patter i specified), the codec is adding the logstash header at the beginning.

thanks again

8

An input will process lines in order, the pipeline will not unless pipeline.workers is set to 1 and pipeline.ordered is set to true (which is the default in v7 when pipeline.workers is set to 1).

9

Ok, i tested it and reducing the workers to 1 logstash follow correctly the order...it will probably result in a loss of performance, right?
Anyway how can i remove the header added by multiline?
Here an example:

Input:


27-Nov-2020 08:53:38.689 SEVERE [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.core.StandardContext.reload Exception starting Context with name [/examples]
	org.apache.catalina.LifecycleException: Failed to start component [org.apache.catalina.webresources.StandardRoot@728b92be]
		at org.apache.catalina.util.LifecycleBase.handleSubClassException(LifecycleBase.java:440)
		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:198)
		at org.apache.catalina.core.StandardContext.resourcesStart(StandardContext.java:4803)

Output:


27-Nov-2020 08:53:38.689 SEVERE [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.core.StandardContext.reload Exception starting Context with name [/examples]2021-03-18T16:20:34.337Z fccapsiem02 	org.apache.catalina.LifecycleException: Failed to start component [org.apache.catalina.webresources.StandardRoot@728b92be]2021-03-18T16:20:34.348Z fccapsiem02 		at org.apache.catalina.util.LifecycleBase.handleSubClassException(LifecycleBase.java:440)2021-03-18T16:20:34.359Z fccapsiem02 		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:198)2021-03-18T16:20:34.369Z fccapsiem02 		at org.apache.catalina.core.StandardContext.resourcesStart(StandardContext.java:4803)2021-03-18T16:20:34.375Z fccapsiem02

I want to remove the header (2021-03-18T16:20:34.375Z fccapsiem02)....

Thanks again

10

You could do that using mutate+gsub.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.