Timestamp pattern not working for logstash filtering

I have 3 lines of logs with different structure and i am constructing a grok pattern to filter the logs. But the pattern I have isn't working. Even when I use just the timestamp pattern, it doesn't match.

[2022-10-04 21:45:27,444: INFO/MainProcess] Events of group {task} enabled by remote
[2022-10-04 21:43:06,521: ERROR/MainProcess] consumer: Cannot connect to redis:// Error 111 connecting to Connection refused..
[2022-10-04 21:45:22 +0000] [3094] [INFO] Listening at: (3094)

I am expecting:


But as I mentioned, even when I use just the timestamp pattern below to test on grokdebugger, it doesn't match for any of the logs:

This is the current pattern i have:
\[%{TIMESTAMP_ISO8601:timestamp}\]\:%{LOGLEVEL:loglevel}%{WORD: class} %{SPACE}%{GREEDYDATA:logMessage}

Your grok pattern has a closing ] on the timestamp But your logs do not on the first two lines.
3rd Line is completely different

Perhaps start with these 2 patterns and build from there

\[%{TIMESTAMP_ISO8601:timestamp}: %{LOGLEVEL:loglevel}/%{WORD:class}\] %{GREEDYDATA:logMessage}
\[%{TOMCAT_DATESTAMP:timetamp}\] \[%{POSINT:bytes}\] \[%{LOGLEVEL:loglevel}\] %{GREEDYDATA:logMessage}

Not sure if that 3094 is bytes etc...

Highly Recommend this tool incremental construction

  "class": "MainProcess",
  "loglevel": "INFO",
  "logMessage": "Events of group {task} enabled by remote",
  "timestamp": "2022-10-04 21:45:27,444"

  "class": "MainProcess",
  "loglevel": "ERROR",
  "logMessage": "consumer: Cannot connect to redis:// Error 111 connecting to Connection refused..",
  "timestamp": "2022-10-04 21:43:06,521"

  "timetamp": "2022-10-04 21:45:22 +0000",
  "bytes": "3094",
  "loglevel": "INFO",
  "logMessage": "Listening at: (3094)"

When / If you use the date filter to convert to a date field you will need to provide matching time patterns... what this does is just extract the timestamp as a string.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.