Hi,
I've just started playing with Logstash 2.0.0 and so far it looks great
And it is working fine for single line entries, but not for multiline entries.
Here is my config file to pass the file.
input {
file {
path => "F:/test.log"
start_position => beginning
codec => multiline {
pattern => "^%{TIMESTAMP_ISO8601} "
negate => true
what => previous
}
}
}
filter {
grok {
patterns_dir => "./patterns"
match => {
# - %{MY_LOGMESSAGE:logmessage}
"message" => ["%{TIMESTAMP_ISO8601:timestamp} [%{MY_THREAD_NAME:thread}] %{MY_LOGLEVEL:loglevel} %{JAVACLASS:class} - %{MY_LOGMESSAGE:logmessage}"]
}
}
}
output {
stdout { codec => json }
}
Here is the pattern file that I'm currently using:
MY_THREAD_NAME ([^]]+)
MY_LOGLEVEL ([Tt]race|TRACE|[Dd]ebug|DEBUG|[Ii]nfo|INFO|[Ww]arn|WARN|[Ee]rror|ERROR|)\s*
MY_CLASSNAME (?:[a-zA-Z0-9/-]+.)+[A-Za-z0-9$]+
MY_LOGMESSAGE ((.|\s)*)
I've been using https://regex101.com to ensure that I've got my pattern correct which is as follows:
(?>\d\d){1,2}-(?:0?[1-9]|1[0-2])-(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])T:(?:[0-5][0-9]):(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?) [([^]]+)] (([Tt]race|TRACE|[Dd]ebug|DEBUG|[Ii]nfo|INFO|[Ww]arn|WARN|[Ee]rror|ERROR|)\s*) ((?:[a-zA-Z0-9/-]+.)+[A-Za-z0-9$]+) - ((.|\s)*)
Here is a sample of a single line entry:
2015-11-18T09:41:10,436 [pool-4-thread-1 ] INFO com.test.Application - Starting Application
This is the json that is printed to the screen (with some formatting applied):
{
"@timestamp": "2015-11-18T09:41:10.628Z",
"message": "2015-11-18T09:41:10,436 [pool-4-thread-1 ] INFO com.test.Application - Starting Application\r",
"@version": "1",
"host": "localhost",
"path": "F:/test.log",
"timestamp": "2015-11-18T09:41:10,436",
"thread": "pool-4-thread-1 ",
"loglevel": "INFO ",
"class": "com.test.Application",
"logmessage": "Starting Application\r"
}