Grokdebug says expression is ok but I get grokfailures when running live

[bhu73 (mark)]

I have a mangled syslogish message format coming from Microsoft Orchestrator :-

{"message":"<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhnÄÄÄ\u0000","@version":"1","@timestamp":"2019-03-25T10:47:07.173Z","host":"192.168.248.50","type":"syslog"}

Grokdebugger is happy with this

(?\d+)>(?.+)\.+(?\d+).+%{TIMESTAMP_ISO8601:timestamp}.+%{IPORHOST:device}

logstash looks like this

filter {
grok {
match => [ "message" , "(?\d+)>(?.+)\.+(?\d+).+%{TIMESTAMP_ISO8601:timestamp}.+%{IPORHOST:device}" ]
}
}

and I get grokfailures and no fields in ES. Clearly I am doing something stupid. Would someone care to point out what.?

[A_B]

@bhu73

"message":"<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhnÄÄÄ\u0000"

message is only that ^^ at least as far as I can tell...

Testing with jq .

$ jq .
{"message":"<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhnÄÄÄ\u0000","@version":"1","@timestamp":"2019-03-25T10:47:07.173Z","host":"192.168.248.50","type":"syslog"}
    {
      "message": "<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhnÄÄÄ\u0000",
      "@version": "1",
      "@timestamp": "2019-03-25T10:47:07.173Z",
      "host": "192.168.248.50",
      "type": "syslog"
    }

As you have JSON coming in you don't really have to use grok for timestamp and device.

Coolt testmeddelande

[bhu73 (mark)]

@bhu73

{"message":"<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhnÄÄÄ\u0000","@version":"1","@timestamp":"2019-03-25T10:47:07.173Z","host":"192.168.248.50","type":"syslog"}

[WARN ] 2019-03-26 10:53:51.298 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>"<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhnÄÄÄ\u0000", :exception=>#<LogStash::Json::ParserError: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: (byte)"<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhn���"; line: 1, column: 2]>}

_jsonparsefailure

[A_B]

Can you set the input to { codec => "json" }?

If not, you would have to tell Logstash what to look for in the filter section. Here is an example. My sample config is first and then the result. This is reading the input as line (not json).

# cat ls-json.conf
input { stdin { } }

filter {
  json {
    source => "message"
  }
}
output {
  stdout { codec => rubydebug }
}
# logstash-6.3.1/bin/logstash -f ls-json.conf
Sending Logstash's logs to /root/tmp/logstash-6.3.1/logs which is now configured via log4j2.properties
[2019-03-26T10:53:48,547][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-03-26T10:53:48,653][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.3.1"}
[2019-03-26T10:53:48,923][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>12, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-03-26T10:53:48,945][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x52c5a4b@/root/tmp/logstash-6.3.1/logstash-core/lib/logstash/pipeline.rb:245 sleep>"}
The stdin plugin is now waiting for input:
[2019-03-26T10:53:48,954][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-03-26T10:53:49,001][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9601}
{"message":"<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhnÄÄÄ\u0000","@version":"1","@timestamp":"2019-03-25T10:47:07.173Z","host":"192.168.248.50","type":"syslog"}
{
          "host" => "192.168.248.50",
          "type" => "syslog",
      "@version" => "1",
    "@timestamp" => 2019-03-25T10:47:07.173Z,
       "message" => "<41>Testmeddelande: jhkhuhuihuhukhukjhkjhkjhnÄÄÄ\u0000"
}
...