Logstash Filter Verifier - Multiline Messages

[dawi]

Logstash Filter Verifier works great for testing even complex Logstash Pipelines.

However it seems that it currently does not support testing of multiline messages properly.

Whenever I add a "\n" into the message string, it will increase the number of events, which will cause the tests to fail like this:

Testcase failed, continuing with the rest: Expected 3 event(s), got 5 instead.

How can I test multiline messages (eg. messages including Java StackTraces) via Logstash Filter Verifier?

@magnusbaeck 1. Is this currently possible? 2. Thanks for this great convenient tool.

[dawi]

Ok, i figured out that it is perfectly possible to test multiline messages with logstash filter verifier.
To do so, the codec json has to be used instead of line or json_line.

One thing that drove me crazy is that if you use the --sockets flag (what you very likely want to speed up the tests) you cannot have more than one multiline test case per test file.

In this case you currently have two options:

1. Don't use the --sockets flag (which will result in slow tests)
2. Put each multiline test case in a separate file.

Maybe I should make a bugreport, but I am not sure if it is a bug or intended behaviour.

[antoinepourriot]

Hi @dawi

i struggle to test multiline events. Could you please share a sample test file ?

mine is:
"codec": "json",
"input": [
"{"message": "13:59:17,601 WARN [org.apache.cxf.phase.PhaseInterceptorChain] Interceptor for {http://proofofidentity.documents.fr/}ProofOfIdentityWebService has thrown exception, unwinding now\nmiddleware.exceptions.TechnicalException: Some constraints\n\t The field value has value \n\n\tat java.lang.Thread.run(Thread.java:745)"}"
],

ouput is :
Testcase failed, continuing with the rest: Expected 1 event(s), got XX instead.

[dawi]

Hi, Antoine. You should not be using the json codec. Testing Multiline Messages works fine with the json_lines codec. See multiple multiline messages testcases don't work with --sockets flag · Issue #39 · magnusbaeck/logstash-filter-verifier · GitHub for more information.

Note that you have to double escape the linebreaks. Example:

"{"message":"line 1\\nline 2\\nline 3"}"

[antoinepourriot]

@dawi thanks a lot !