How to parse Elasticsearch logs?

(Punit Naik) #1

I have an elasticsearch log and one line from that looks like this:

[2016-02-02 00:03:17,558][INFO ][cluster.service ] [Sugar Man] added {[factordb-d0b9dfee4570f86e--3b3c30d8-1529e1c2cec--8000][5ayv3rbgQT6AK7iWoy3u7g][fc-machine][inet[/]]{data=false, master=false},}, reason: zen-disco-receive(join from node[[factordb-d0b9dfee4570f86e--3b3c30d8-1529e1c2cec--8000][5ayv3rbgQT6AK7iWoy3u7g][fc-machine][inet[/]]{data=false, master=false}])

Can anyone help me to find the grok pattern for this line?


(Punit Naik) #2

I already found the answer:

match => { "message"=> "\[%{TIMESTAMP_ISO8601:log_timestamp}\]\[%{DATA:log_level}%{SPACE}\]\[%{DATA:source}%{SPACE}\]%{SPACE}\[%{DATA:node}\]%{SPACE}%{GREEDYDATA:log_message}"}

(Magnus B├Ąck) #3

Watch out for multiple DATA and GREEDYDATA patterns in the same expression. You might get surprising matches.

(Punit Naik) #4

I tested the filter and it worked just fine. Thanks for the advice though Mr. @magnusbaeck :slightly_smiling:

(system) #5