Parsing F5 LTM logs with Logstash

[sarahvo]

Hi, I'm trying to parse LTM logs using this example (found in the official documentation) below:

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

I'm using filebeat to ship the logs to logstash, however the parsing doesn't seem to be working. Any tips or help would be greatly appreciated!

Here's an example of how my logs currently look below:

[Badger]

Please do not post pictures of text, just post the text. What does the [message] field look like?

[sarahvo]

The message field looks like this:

message 2019-08-29T10:45:01-07:00 devlkfltm02 info CROND[12075]: (root) CMD (nice -n 19 ionice -c 3 /usr/share/ts/bin/asm_logrotate)

[Badger]

The timestamp is nothing like a SYSLOGTIMESTAMP, which would be something like "Aug 29 12:34:56". Replace SYSLOGTIMESTAMP with TIMESTAMP_ISO8601.

You also need to modify the date filter to parse the format you have.

match => [ "syslog_timestamp", "ISO8601" ]

[sarahvo]

Okay, thanks. Is there any documentation where I can find how to properly format this?

[Badger]

The documentation for the patterns pre-defined for grok is really the patterns themselves. On my system they are in a set of files in the directory /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-patterns-core-4.1.2/patterns/

[sarahvo]

Okay, I'm going to try and fix the grok patterns. I'm running elasticsearch, logstash, and kibana using docker-compose. Do you know if it's sufficient just to restart the logstash container for the updates on the configuration to take place?

[Badger]

@sarahvo

Do you know if it's sufficient just to restart the logstash container for the updates on the configuration to take place?

Yes, and if you run with --config.reload.automatic on the command line you do not even need to restart.