GROK matches nothing. Ever

Elastic Stack Logstash
1

Hi,

I am new to logstash and want to read to read FHEM logs.

A logline looks like this:
2016-10-08_22:06:23 Sensor_Garden T: 7.9 H: 89

This is my logstash configuration:

input {
file {
path => [ "c:/Java/elk/fhem-logs/test/*.log" ]
start_position => beginning
ignore_older => 0
type => "fhem"
}
}
filter {
if [type] == "fhem" {
grok {
match => { "message" => "%{DATA:logdate} %{DATA:device} T: %{NUMBER:temperatur:float} H: %{NUMBER:humidity:float}" }
}
date {
match => [ "logdate", "YYYY-MM-dd_HH:mm:ss" ]
timezone => "Europe/Berlin"
target => "logdate"
}
}
}
output {
#elasticsearch {
# hosts => [ "localhost:9200" ]
#}
stdout {
codec => rubydebug
}
}

test.config (logstash-5.0.0-beta1\bin\logstash.bat --path.config fhem.conf --config.test_and_exit) says "Configuration OK"

The output I get is:
{
"path" => "c:/Java/elk/fhem-logs/test/Sensor_Garden-2016.log",
"@timestamp" => 2016-10-11T09:12:08.911Z,
"@version" => "1",
"host" => "SV-NBK-636",
"message" => "2016-01-01_06:58:10 Sensor_Garden T: 4.4 H: 99",
"type" => "fhem",
"tags" => [
[0] "_grokparsefailure"
]
}

So I tried my expression with http://grokdebug.herokuapp.com/ but I only get "no match". Even if I try %{WORD:something} with input abc I get "no match". I am lost here.

2

Works for me:

$ cat test.config 
input { stdin { } }
output { stdout { codec => rubydebug } }
filter {
  grok {
    match => {
      "message" => "%{DATA:logdate} %{DATA:device} T: %{NUMBER:temperatur:float} H: %{NUMBER:humidity:float}"
    }
  }
}
$ echo '2016-10-08_22:06:23 Sensor_Garden T: 7.9 H: 89' | logstash -f test.config  
Settings: Default pipeline workers: 8
Pipeline main started
{
       "message" => "2016-10-08_22:06:23 Sensor_Garden T: 7.9 H: 89",
      "@version" => "1",
    "@timestamp" => "2016-10-11T09:57:12.929Z",
          "host" => "lnxolofon",
       "logdate" => "2016-10-08_22:06:23",
        "device" => "Sensor_Garden",
    "temperatur" => 7.9,
      "humidity" => 89.0
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}

I wonder if the problem could be the trailing carriage return (since you're running on Windows). What if you add \s* to the end of the grok expression?

3

I am sorry, the solution is too simple to be true. There is a double blank in my GROK expression.

I still do not understand why http://grokdebug.herokuapp.com/ always says "No Match" - that would have helped to track that down - but it works now. Thanks a lot for your help.

4

See, this is why it's important to always post configuration snippets and logs formatted as preformatted text.

1 Like