How to parse Elasticsearch logs?

Elastic Stack Logstash
1

I have an elasticsearch log and one line from that looks like this:

[2016-02-02 00:03:17,558][INFO ][cluster.service ] [Sugar Man] added {[factordb-d0b9dfee4570f86e--3b3c30d8-1529e1c2cec--8000][5ayv3rbgQT6AK7iWoy3u7g][fc-machine][inet[/192.168.2.9:9301]]{data=false, master=false},}, reason: zen-disco-receive(join from node[[factordb-d0b9dfee4570f86e--3b3c30d8-1529e1c2cec--8000][5ayv3rbgQT6AK7iWoy3u7g][fc-machine][inet[/192.168.2.9:9301]]{data=false, master=false}])

Can anyone help me to find the grok pattern for this line?

Thanks!

2

I already found the answer:

match => { "message"=> "\[%{TIMESTAMP_ISO8601:log_timestamp}\]\[%{DATA:log_level}%{SPACE}\]\[%{DATA:source}%{SPACE}\]%{SPACE}\[%{DATA:node}\]%{SPACE}%{GREEDYDATA:log_message}"}
3

Watch out for multiple DATA and GREEDYDATA patterns in the same expression. You might get surprising matches.

1 Like
4

I tested the filter and it worked just fine. Thanks for the advice though Mr. @magnusbaeck :slightly_smiling: