Multiline is split from main log message - Any help please

Elastic Stack Beats
1

Here is a snippet of the raw log output:

[2020-07-15T08:21:58+02:00] 172.16.35.104 ERROR Exception: <log:context>{"exception":"[object] (App\\Action\\Exception\\MethodNotAllowedException(code: 405): There is no action defined for this method and this URI. at /home/bike24/products/services/api/releases/2.3.3/src/App/Middleware/MethodNotAllowedResponseHandler.php:30)
[stacktrace]
#0 /home/app/products/services/api/releases/2.3.3/vendor/zendframework/zend-expressive/src/Middleware/LazyLoadingMiddleware.php(80): App\\Middleware\\MethodNotAllowedResponseHandler->__invoke(Object(Zend\\Diactoros\\ServerRequest), Object(Zend\\Diactoros\\Response), Object(Closure))
#1 /home/app/products/services/api/releases/2.3.3/vendor/zendframework/zend-stratigility/src/Next.php(128): Zend\\Expressive\\Middleware\\LazyLoadingMiddleware->process(Object(Zend\\Diactoros\\ServerRequest), Object(Zend\\Stratigility\\Next))

My filebeat configuration:

    ---
    filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - /var/log/nginx/*.log
        - /var/log/app/*.log

      exclude_files: ['\.gz$']
      multiline.pattern: '^\[|^\d{4}\/\d{2}\/\d{2}'
      multiline.negate: true
      multiline.match: after

    filebeat.config.modules:
      # Glob pattern for configuration loading
      path: ${path.config}/modules.d/*.yml
      
      # Set to true to enable config reloading
      reload.enabled: false

      # Period on which files under path should be checked for changes
      #reload.period: 10s

    setup.template.settings:
      index.number_of_shards: 3
      #index.codec: best_compression
      #_source.enabled: false

    name: "app-1-filebeat"
    tags: ["app-1"]

    output.logstash:
      hosts: ["mon-1.server.net:2561"]
      ssl.certificate_authorities: ["/usr/share/ca-certificates/app/Webserver.crt"]
      ssl.certificate: "/etc/ssl/certs/server.net.crt"
      ssl.key: "/etc/ssl/private/server.net.key"
      ssl.verification_mode: none

Here is my logstash config:

grok pattern:

`\[%{TIMESTAMP_ISO8601:b24_timestamp}\] %{IP:ClientIP} %{LOGLEVEL:loglevel} %{GREEDYDATA:log_message}`

input:

    input {
      beats {
      port => 2561
      ssl => true
      ssl_certificate => "/etc/ssl/certs/server.crt"
      ssl_key => "/etc/ssl/private/server.p8"
      }
    }

output:

       elasticsearch {
          hosts => "localhost:9200"
          index => "var_log_app-%{+YYYY.MM.dd}"
        }

So when I visualise the output in Kibana the message and stacktrace are split into different rows in the discovery tab.

Can you see anything wrong with my config that would make this happen or what can I do to check what is actually happening? By the way the logs do not appear so often by they are error logs so I only have data from early this morning.

2

Anyone?

3

If you have a question about the filebeat multiline matching functionality then I suggest you move that to the filebeat forum.

4

Thanks Badger I did that

5

I fixed this by changing the pattern in the multiline pattern. ^[\d

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.