Filebeat multiline some pattern

Elastic Stack Beats
1

There is a tomkat application in the docker and I want to set up logging in elk, but the problem is that the logs have a different format

2018-11-16 13:23:07 [http-nio-127.0.0.1-8080-exec-21] DEBUG c.j.m.s.filter.AgeConfirmFilter - ### Previous url: http://example.com/
2018/11/16 08:23:07 [error] 31117#31117: *3617005 open() "/app/tomcat/webapps/example/apple-touch-icon.png" failed (2: No such file or directory), client: 172.17.0.1, server: example.com, request: "GET /apple-touch-icon.png HTTP/1.0", host: "example"
2018/11/16 08:23:07 [error] 31117#31117: *3617005 could not find named location "@fallback", client: 172.17.0.1, server: example.com, request: "GET /apple-touch-icon.png HTTP/1.0", host: "example.com"
2018-11-16 13:23:13 [http-nio-127.0.0.1-8080-exec-32] DEBUG c.j.m.s.filter.AgeConfirmFilter - Age confirmation: true
Fri Nov 16 13:23:13 UTC 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.
5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing
 applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL
=false, or set useSSL=true and provide truststore for server certificate verification.
2018-11-16 13:23:15 [http-nio-127.0.0.1-8080-exec-83] DEBUG c.j.m.s.filter.AgeConfirmFilter - Age confirmation: true
250 2.0.0 Ok: queued as 2BEF93600C4
DEBUG SMTP: message successfully delivered to mail server
QUIT
221 2.0.0 Bye
2018-11-16 13:23:15 [http-nio-127.0.0.1-8080-exec-55] DEBUG c.j.m.helpers.EmailHelper - MAIL WAS SENT, manage post

configuration filebeat.yml, I tried to add two pattern, but it does not work correctly

filebeat.inputs:
- type: docker
  containers.ids:
    - "*"
  multiline:
    pattern: '^([0-9]{4}-[0-9]{2}-[0-9]{2})|([J|F|M|A|M|S|O|N|D][a-z]{2} [0-9]{1,2}, [0-9]{2}) '
    pattern: '^([0-9]{4}/[0-9]{2}/[0-9]{2})|([J|F|M|A|M|S|O|N|D][a-z]{2} [0-9]{1,2}, [0-9]{2}) '
    negate: true
    match: before

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 3

output.logstash:
  hosts: ["logstash_server:5044"]

any help in my question, I will be very grateful

2

maybe someone come in handy, did so

filebeat.inputs:
- type: docker
  containers.ids:
    - "*"
  multiline:
    pattern: '^([0-9]{4}[/-][0-9]{2}[/-][0-9]{2}\s+[0-9]{2}\:[0-9]{2}\:[0-9]{2})|^([MTWFS][a-z]{2}\s+[JFMASOND][a-z]{2}\s+[0-9]{2}\s+[0-9]{2}\:[0-9]{2}\:[0-9]{2})'
    negate: true
    match: after

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 3

output.logstash:
  hosts: ["logstash_server:5044"]
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.