XML message in catalina log

[Wayne_Taylor]

Hi Team,

I have my catalina.out parsed but once exception it misses XML over multi line. I need some help to get this working.

Below is my config thus far:

filebeat.prospectors:
- document_type: logs
  paths:
  - /Users/wtaylor/Downloads/logstash-5.3.0/bin/example.txt

multiline:
- pattern: "^20%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}"
- negate: true
- match: after

output.logstash:
 hosts: ["127.0.0.1:3335"]

Below is an example of what is matching and what isn't

Match:

{
        "server" => "prod1",
    "@timestamp" => 2017-09-01T23:06:42.342Z,
        "offset" => 1807,
      "@version" => "1",
    "input_type" => "log",
          "beat" => {
        "hostname" => "MB-C02TF1QUGTFM",
            "name" => "MB-C02TF1QUGTFM",
         "version" => "5.5.2"
    },
          "host" => "MB-C02TF1QUGTFM",
        "source" => "/Users/wtaylor/Downloads/logstash-5.3.0/bin/example.txt",
       "message" => "2017-08-31T14:25:26.074|INFO|ACPU2|ABP|com.aircell.shared.web.servlet.SessionInfoLoggingFilter.doFilter:39|Id:_172.19.131.144|Thread:http-nio-1000-exec-12|SessionInfoLoggingFilter  :: doFilter",
          "type" => "logs",
          "tags" => [
        [0] "beats_input_codec_plain_applied"
    ]
}

Miss:

XML Object in log over multi lines:

<java version="1.8.0_65" class="java.beans.XMLDecoder">
 <object class="com.aircell.abp.model.FlightInformation">
    <void property="VersionNo">
   <string>P15340_RevKK</string>
  </void>
  <void property="aircraftTailNumber">
   <string>N270AK</string>
  </void>
 </object>
</java>

Any help greatly appreciated.

Wayne

[steffens (Steffen Siering)]

You have an error in your filebeat configuration. The multiline settings are per prospector, not global. Indenting all multiline settings by 2 spaces should help.

The syntax used in the pattern setting is not correct. One can only use plain regular expressions. Also %{<name>} is not supported by beats. Try: '%20\d{2}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}'. Don't use double quotes for regular expressions in YAML configuration files.

With multiline one tries to capture the 'structure' of the log, not the contents so to say. If you're still having problem please include some more consecutive log lines.

[Wayne_Taylor]

Hi @steffens, thanks but same issue.

Here is my config as per request:
filebeat.prospectors:
- document_type: abp_logs
paths:
- /Users/wtaylor/Downloads/logstash-5.3.0/bin/example.txt

multiline:
- pattern: '%20\d{2}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}'
- negate: true
- match: after

output.logstash:
 hosts: ["127.0.0.1:3335"]![02 AM|690x375](upload:/

Log Messages attached - sorry but the XML gets truncated so needed as image

Line 6 where the XML begins shows and then each line of the xml shows

[steffens (Steffen Siering)]

Please, copy and paste contents. No screenshots. Not everyone might be able to see your screenshots + I'm not willing to type out the screenshot for testing.

Check the actual date pattern in your screenshot and the regular expression you have configured. They clearly do not match. Consider: '^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{3}\|'.

For tips on testing multiline also check the filebeat documentation.