Problem with filebeat, it does not process multiline correctly

Elastic Stack Elastic Agent
1

Hello.
I have a problem setting up the filebeat agnet.
The logs consist of several lines and I would like to send them as one message.
I prepared the config, but for some reason it doesn’t work. tell me what the problem is?
When writing a regular expression, I used https://regex101.com. Everything looks good there, but the messages are still coming in the wrong form.

Parsing example

image
image839×627 103 KB

Part of the config from filebeat 8.12.1

- type: filestream
  id: cc-vapp-oracle-attention
  tags: ["oracle-attention"]
  enabled: true
  ignore_older: 24h
  paths:
       - /opt/oracle/diag/rdbms/xe/XE/trace/attention_XE_Test*.log
  parsers:
    - multiline:
       type: pattern
       pattern: '^{'
       negate: true
       match: after
       flush_pattern: '^}'

and

- type: filestream
  id: cc-vapp-oracle-attention
  tags: ["oracle-attention"]
  enabled: true
  ignore_older: 24h
  paths:
       - /opt/oracle/diag/rdbms/xe/XE/trace/attention_XE_Test*.log
  parsers:
    - multiline:
       type: pattern
       pattern: ^\n[[:graph:]]\n  "NOTIFICATION"'
       negate: true
       match: after
       flush_pattern: '^}'

example log:

{
  "NOTIFICATION" : "Starting ORACLE instance (normal) (OS id: 30427)",
  "URGENCY"      : "INFO",
  "INFO"         : "Additional Information Not Available",
  "CAUSE"        : "A command to startup the instance was executed",
  "ACTION"       : "Check alert log for progress and completion of command",
  "CLASS"        : "CDB Instance / CDB ADMINISTRATOR / AL-1000",
  "TIME"         : "2022-11-12T04:45:47.265+02:00"
}

{
  "NOTIFICATION" : "Shutting down ORACLE instance (immediate) (OS id: 30720)",
  "URGENCY"      : "INFO",
  "INFO"         : "Shutdown is initiated by sqlplus (TNS V1-V3). ",
  "CAUSE"        : "A command to shutdown the instance was executed",
  "ACTION"       : "Check alert log for progress and completion of command",
  "CLASS"        : "CDB Instance / CDB ADMINISTRATOR / AL-1001",
  "TIME"         : "2022-11-12T04:46:53.327+02:00"
}

{
  "NOTIFICATION" : "Starting ORACLE instance (normal) (OS id: 30723)",
  "URGENCY"      : "INFO",
  "INFO"         : "Additional Information Not Available",
  "CAUSE"        : "A command to startup the instance was executed",
  "ACTION"       : "Check alert log for progress and completion of command",
  "CLASS"        : "CDB Instance / CDB ADMINISTRATOR / AL-1000",
  "TIME"         : "2022-11-12T04:46:54.258+02:00"
}
2

Hello @San9

I tested the configuration below and it works fine.

- type: filestream
  id: my-filestream-id
  enabled: true
  paths:
    - example.txt
  parsers:
  - multiline:
      type: pattern
      pattern: '^\{'
      negate: true
      match: after
1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.