Logging pattern configured in log4js.json is below:
"appenders": [
{
"type" : "logLevelFilter",
"level" : "INFO",
"appender": {
"type": "common-log.syslog",
"tag": "AM-GUI",
"facility": "user",
"address": "localhost",
"port": 500,
"layout": {
"type": "pattern",
"pattern": "%p;%d{yyyy-MM-ddThh:mm:ss.SSSO};%x{process};%x{agent};%m;%x{uri};%x{arguments};%x{result};%x{operation};%x{source};%x{duration};%x{errorCode}"
}
}
},
Actual log file is:
Apr 15 06:37:53 hostname SYSLOG 8115 - - INFO;2016-04-15T09:37:53.235+0300;DCM-GUI;username;Received request to retrieve a list of deposits from Data Service;/?,false;-;-;READ;log.js;-;-
Apr 15 06:37:58 hostname SYSLOG 8115 - - INFO;2016-04-15T09:37:58.356+0300;DCM-GUI;username;Received request to retrieve a list of deposits from Data Service;,false;-;-;READ;log.js;-;-
Apr 15 06:38:02 hostname SYSLOG 7196 - - INFO;2016-04-15T09:38:02.303+0300;CMM;-;No blacklist items at all;-;-;-;-;blacklistRest.js;-;-
Apr 15 06:38:11 hostname SYSLOG 7194 - - INFO;2016-04-15T09:38:11.871+0300;CMM;-;No blacklist items at all;-;-;-;-;blacklistRest.js;-;-
Apr 15 06:38:11 hostname SYSLOG 7196 - - INFO;2016-04-15T09:38:11.877+0300;CMM;-;No blacklist items at all;-;-;-;-;blacklistRest.js;-;-
Apr 15 06:38:25 hostname SYSLOG 7194 - - INFO;2016-04-15T09:38:25.683+0300;CMM;-;There is no blacklist entry for '123456';-;-;-;-;blacklistRest.js;-;-
Apr 15 06:38:37 hostname SYSLOG 7194 - - INFO;2016-04-15T09:38:37.725+0300;CMM;-;No blacklist items at all;-;-;-;-;blacklistRest.js;-;-
When I see those logs in Kibana, those are clubbed together in one docuemnt with tag multiline.
I am using multiline code under input redis as below.
input {
redis {
host => "localhost"
data_type => "list"
key => "logstash"
threads => 20
batch_count => 2000
codec => multiline {
pattern => "^\s"
what => "previous"
negate => true
}
}
}
filter{
multiline {
pattern => "^**%{TIMESTAMP_ISO8601}** "
what => "previous"
negate => true
}
Should I use some better pattern in multiline filter above? What should be that pattern if we assume given configuration of log4js.json is correct?
br,
Sunil