Logstash Grok Filter breaks Single log line as 2 lines

Elastic Stack Logstash
1

I am working to write a simple grok pattern and I have generated a grok filter which works awsome in grok debugger but when I use it in logstash, it generates 2 lines out of single line of log. these message also not propagating to ES.

I am using Filebeat[5.3.0]-->Logstash[5.3.0]-->ElasticSearch[5.3.0]

Log Line :-

[WARN ] 2017-08-29 16:42:24.771 [http-0.0.0.0:8042-3] PageNotFound - Request method 'POST' not supported

Grok Filter :-

[%{LOGLEVEL:LOG_LEVEL}\ ] %{TIMESTAMP_ISO8601:timestamp_match} %{GREEDYDATA:message}

Logstash processed message :-

{
"offset" => 9438,
"timestamp_match" => "2017-09-25 20:02:18.771",
"input_type" => "log",
"source" => "/my/source/logfile.log",
"message" => [
[0] "[WARN ] 2017-09-25 20:02:18.771 [http-0.0.0.0:8042-2] PageNotFound - Request method 'POST' not supported",
[1] " [http-0.0.0.0:8042-2] PageNotFound - Request method 'POST' not supported"
** ],**
"type" => "log",
"env" => "UAT",
"tags" => [
[0] "uat",
[1] "beats_input_codec_plain_applied"
],
"log_type" => "spring-logs",
"component" => "App",
"@timestamp" => 2017-09-25T20:02:18.771Z,
"@version" => "1",
"beat" => {
"version" => "5.3.0"
},
"host" => "mu-host",
"LOG_LEVEL" => "WARN"
}

Does someone know why would this happen?

2

I believe the reason is that you are capturing the final GREEDYDATA pattern into a field that already exists (message) without setting overwrite to true.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.