Multiline Codec Error?

Elastic Stack Logstash
1

When i parsing the logs on TIMESTAMP field of multiline codec it is showing error output for the TIMESTAMP fields present in square brackets.

Config:

input {
  file {
    path => "D:\logstash\logstash-2.4.0\bin\slowlogs.txt"
    start_position => "beginning"
    codec => multiline {
        pattern => "^%{TIMESTAMP_ISO8601} "
        negate => true
        what => previous
    }
  }
}

output {
  stdout { codec => rubydebug }
}

Logs:

[2015-08-24 11:49:14,389] [INFO ][env                      ] [Letha] using [1] data paths, mounts [[/
(/dev/disk1)]], net usable_space [34.5gb], net total_space [118.9gb], types [hfs]
[2015-08-24 11:49:14,389] [INFO ][env                      ] [Letha] using [1] data paths, mounts [[/
(/dev/disk1)]], net usable_space [34.5gb], net total_space [118.9gb], types [hfs]
[2015-08-24 11:49:14,389] [INFO ][env                      ] [Letha] using [1] data paths, mounts [[/
(/dev/disk1)]], net usable_space [34.5gb], net total_space [118.9gb], types [hfs]

Output:

{
    "@timestamp" => "2017-05-23T11:19:10.635Z",
       "message" => "[2015-08-24 11:49:14,389] [INFO ][env
] [Letha] using [1] data paths, mounts [[/\r\n(/dev/disk1)]], net usable_space [
34.5gb], net total_space [118.9gb], types [hfs]\r\n[2015-08-24 11:49:14,389] [IN
FO ][env                      ] [Letha] using [1] data paths, mounts [[/\r\n(/de
v/disk1)]], net usable_space [34.5gb], net total_space [118.9gb], types [hfs]\r\
n[2015-08-24 11:49:14,389] [INFO ][env                      ] [Letha] using [1]
data paths, mounts [[/\r\n(/dev/disk1)]], net usable_space [34.5gb], net total_s
pace [118.9gb], types [hfs]\r\n\r\n\r",
      "@version" => "1",
          "tags" => [
        [0] "multiline"
    ],
          "path" => "D:\\logstash\\logstash-2.4.0\\bin\\slowlogs.txt",
          "host" => "PC326815"
}

The same logs removing it from []

2015-08-24 11:49:14,389 [INFO ][env                      ] [Letha] using [1] data paths, mounts [[/
(/dev/disk1)]], net usable_space [34.5gb], net total_space [118.9gb], types [hfs]
2015-08-24 11:49:14,389 [INFO ][env                      ] [Letha] using [1] data paths, mounts [[/
(/dev/disk1)]], net usable_space [34.5gb], net total_space [118.9gb], types [hfs]
2015-08-24 11:49:14,389 [INFO ][env                      ] [Letha] using [1] data paths, mounts [[/
(/dev/disk1)]], net usable_space [34.5gb], net total_space [118.9gb], types [hfs]

Executed the same config got output like this:

{
    "@timestamp" => "2017-05-23T11:25:48.075Z",
       "message" => "2015-08-24 11:49:14,389 [INFO ][env                      ]
[Letha] using [1] data paths, mounts [[/\r\n(/dev/disk1)]], net usable_space [3
.5gb], net total_space [118.9gb], types [hfs]\r",
      "@version" => "1",
          "tags" => [
        [0] "multiline"
    ],
          "path" => "D:\\logstash\\logstash-2.4.0\\bin\\slowlogs.txt",
          "host" => "PC326815"
}
{
    "@timestamp" => "2017-05-23T11:25:48.278Z",
       "message" => "2015-08-24 11:49:14,389 [INFO ][env                      ]
[Letha] using [1] data paths, mounts [[/\r\n(/dev/disk1)]], net usable_space [3
.5gb], net total_space [118.9gb], types [hfs]\r",
      "@version" => "1",
          "tags" => [
        [0] "multiline"
    ],
          "path" => "D:\\logstash\\logstash-2.4.0\\bin\\slowlogs.txt",
          "host" => "PC326815"
}
←[33mSIGINT received. Shutting down the agent. {:level=>:warn}←[0m
stopping pipeline {:id=>"main"}
{
    "@timestamp" => "2017-05-23T11:25:57.421Z",
       "message" => "2015-08-24 11:49:14,389 [INFO ][env                      ]
[Letha] using [1] data paths, mounts [[/\r\n(/dev/disk1)]], net usable_space [3
.5gb], net total_space [118.9gb], types [hfs]\r\n\r\n\r",
      "@version" => "1",
          "tags" => [
        [0] "multiline"
    ],
          "path" => "D:\\logstash\\logstash-2.4.0\\bin\\slowlogs.txt",
          "host" => "PC326815"
}

THIS is giving me proper output .How can i achieve the same for the TIMESTAMP field in [] brackets.

I also tried it by giving pattern => "^\[%{TIMESTAMP_ISO8601}\] "
THANKS

2

The pattern is wrong in multiline codec . Worked fine when i used the below config:

input {
  file {
    path => "D:\logstash\logstash-2.4.0\bin\slowlogs.txt"
    start_position => "beginning"
    codec => multiline {
        pattern => "^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]"
        negate => true
        what => previous
    }
  }
}

output {
  stdout { codec => rubydebug }
}
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.