Grok a custom log

Elastic Stack Logstash
1

Hi There I am looking at how to grok a custom log type. At present that is gathering logs in the syslog format with a couple of extra labels.

The following is a log sample.

2017-10-05T14:21:58+01:00 192.168.60.13 random/071b2f1b6074fa33ab763d5945c6b35a45bd47c71ca6df3ad3b2b4622d24377e/sha256: aa40a4a2c71d7b69f61dfd90adfb6be868220b92792de98cf705d54b2ed2ba47/chentex/random-logger:latest[1364]: DEBUG (Thu Oct 5 13:21:58 UTC 2017) first loop completed.
2017-10-05T14:22:00+01:00 192.168.60.13 random/071b2f1b6074fa33ab763d5945c6b35a45bd47c71ca6df3ad3b2b4622d24377e/sha256: aa40a4a2c71d7b69f61dfd90adfb6be868220b92792de98cf705d54b2ed2ba47/chentex/random-logger:latest[1364]: INFO (Thu Oct 5 13:22:00 UTC 2017) takes the value and converts it to string.
2017-10-05T14:22:05+01:00 192.168.60.13 random/071b2f1b6074fa33ab763d5945c6b35a45bd47c71ca6df3ad3b2b4622d24377e/sha256: aa40a4a2c71d7b69f61dfd90adfb6be868220b92792de98cf705d54b2ed2ba47/chentex/random-logger:latest[1364]: ERROR (Thu Oct 5 13:22:05 UTC 2017) something happened in this execution.
2017-10-05T14:22:07+01:00 192.168.60.13 random/071b2f1b6074fa33ab763d5945c6b35a45bd47c71ca6df3ad3b2b4622d24377e/sha256: aa40a4a2c71d7b69f61dfd90adfb6be868220b92792de98cf705d54b2ed2ba47/chentex/random-logger:latest[1364]: INFO (Thu Oct 5 13:22:07 UTC 2017) takes the value and converts it to string.
2017-10-05T14:22:10+01:00 192.168.60.13 random/071b2f1b6074fa33ab763d5945c6b35a45bd47c71ca6df3ad3b2b4622d24377e/sha256: aa40a4a2c71d7b69f61dfd90adfb6be868220b92792de98cf705d54b2ed2ba47/chentex/random-logger:latest[1364]: ERROR (Thu Oct 5 13:22:10 UTC 2017) something happened in this execution.

The pattern that most closely matches is SYSLOG5424BASE my question is how do I collect the following lable into a GROK?

{{.Name}}/{{.FullID}}/{{.ImageFullID}}/{{.ImageName}}

The line above is the data which is being added by the syslog driver how do I add to the existing GROK to capture the extra data? As until now I had avoided writing any custom GROKS.

2

I figured it out so if any one requires something like this then please reference the following

%{TIMESTAMP_ISO8601:Timestamp} %{IP:Host} %{WORD:Docker.Container.Name}[/]%{GREEDYDATA:Docker.Container.Id}[:][\s*]%{WORD:Docker.Container.ImageHash}[/]%{DATA:Docker.Container.Image.Name}[:]%{WORD:Docker.Container.Image.Version}\[%{WORD:system.syslog.pid}\]%{GREEDYDATA:system.syslog.message}

This will take the logs and spit out the message along with a time-stamp and the docker container information.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.