Filebeat multiline pattern for PHP stack trace

[Thessis940]

I am trying to import the PHP FPM logs into an ELK stack. For this I use the filebeat to read the files. Before sending this data to logstash, the multiline log entries should be merged.

For this I built this filebeat configuration:

filebeat.inputs:

  # Each - is an input. Most options can be set at the input level, so
  # you can use different inputs for various configurations.
  # Below are the input specific configurations.

  - type: filestream

    # Change to true to enable this input configuration.
    enabled: true

    # Paths that should be crawled and fetched. Glob based paths.
    paths:
      - '/var/log/app/fpm/*.log'

    multiline.type: pattern
    multiline.pattern: '^\[\d{2}-\w{3}-\d{4} \d{2}:\d{2}:\d{2} [\w/]*\] PHP\s*at.*'
    multiline.negate: false
    multiline.match: after

    processors:
      - add_fields:
          fields.docker.service: "fpm"

But as you can see in the ruby debug output from logstash, the messages were not merged:

{
        "@timestamp" => 2021-08-10T13:54:10.149Z,
        "agent" => {
                 "version" => "7.13.4",
                "hostname" => "3cb76d7d4c7d",
                      "id" => "61dec25e-12ec-4a65-9f1f-ec72a5aa83ee",
            "ephemeral_id" => "631db0d8-60ad-4625-891c-3da09cb0a442",
                    "type" => "filebeat"
        },
             "input" => {
            "type" => "filestream"
        },
               "log" => {
            "offset" => 344,
              "file" => {
                "path" => "/var/log/app/fpm/error.log"
            }
        },
              "tags" => [
            [0] "beats_input_codec_plain_applied",
            [1] "_grokparsefailure"
        ],
            "fields" => {
            "docker" => {
                "service" => "fpm"
            }
        },
          "@version" => "1",
           "message" => "[17-Jun-2021 13:07:56 Europe/Berlin] PHP [WARN] (/var/www/html/Renderer/RendererTranslator.php:92) - unable to translate type integer. It is not a string (/url.php)",
               "ecs" => {
            "version" => "1.8.0"
        }
    }
    {
      "input" => {
            "type" => "filestream"
        },
            "module" => "PHP IES\\ServerException",
               "ecs" => {
            "version" => "1.8.0"
        },
          "@version" => "1",
               "log" => {
            "offset" => 73,
              "file" => {
                "path" => "/var/log/ies/fpm/error.log"
            }
        },
        "@timestamp" => 2021-06-17T11:10:41.000Z,
             "agent" => {
                 "version" => "7.13.4",
                "hostname" => "3cb76d7d4c7d",
                      "id" => "61dec25e-12ec-4a65-9f1f-ec72a5aa83ee",
            "ephemeral_id" => "631db0d8-60ad-4625-891c-3da09cb0a442",
                    "type" => "filebeat"
        },
              "tags" => [
            [0] "beats_input_codec_plain_applied"
        ],
            "fields" => {
            "docker" => {
                "service" => "fpm"
            }
        },
           "message" => "core.login"
    }
    {
        "@timestamp" => 2021-08-10T13:54:10.149Z,
             "agent" => {
                 "version" => "7.13.4",
                "hostname" => "3cb76d7d4c7d",
                      "id" => "61dec25e-12ec-4a65-9f1f-ec72a5aa83ee",
            "ephemeral_id" => "631db0d8-60ad-4625-891c-3da09cb0a442",
                    "type" => "filebeat"
        },
               "ecs" => {
            "version" => "1.8.0"
        },
             "input" => {
            "type" => "filestream"
        },
              "tags" => [
            [0] "beats_input_codec_plain_applied",
            [1] "_grokparsefailure"
        ],
            "fields" => {
            "docker" => {
                "service" => "fpm"
            }
        },
          "@version" => "1",
           "message" => "[17-Jun-2021 13:10:41 Europe/Berlin] PHP     at App\\Module\\ComponentModel\\ComponentModel->doPhase(/var/www/html/Component/Container.php:348)",
               "log" => {
            "offset" => 204,
              "file" => {
                "path" => "/var/log/app/fpm/error.log"
            }
        }
    }

I tested the regular expression with Rubular and it matches the stack trace messages. Also the debug output of Filebeat shows no errors nor warning.

What am I doing wrong here?

[Thessis940]

I was able to solve the problem myself in the meantime. My log files are now written in JSON format so that no more multi-line messages are written.