I am currently trying to use multiline to parse my files. They are inputted into filebeat using filestream.
The logs in the file follow a structured format, except for the first log and last log
[2015-08-24 11:49:14,385] first log
[2015-08-24 11:49:14,389] Start new event #start of log 1
[2015-08-24 11:49:14,395] event 1
[2015-08-24 11:49:14,399] End event #end of log 1
[2015-08-24 11:49:14,403] Start new event
[2015-08-24 11:49:14,407] event 2
[2015-08-24 11:49:14,411] End event
.
.
.
[2015-08-24 11:49:14,988] Start new event
[2015-08-24 11:49:14,992] event 50
[2015-08-24 11:49:14,996] End event
[2015-08-24 11:49:15,000] Last log
I know that I can use a multiline pattern and flush_pattern to parse the structured events learnt here, but what about the first and last log then?
How should I go about using multiline and other techniques to parse this?
Any help is appreciated, thank you!