What is the best way to parse a log file which has sections that I need to remember when reading following lines?


(Andr Eas) #1

Hi everybody,

I'm working on project where I need to parse a log file that is optimized for human consumpion and therefore difficult to parse. It looks like follows

[NODE]-----[DESCRIPTION]-----[DATE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[NODE]-----[DESCRIPTION]-----[DATE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]
[TIME] [PROCESS] [INFORMATION] [OTHERNODE]

[NODE] and [DATE] refer to each of the following lines of the section.

I want to visualize this in Kibana in a way, that allows to link [NODE] and [OTHERNODE]. Apart from that I obviously need to combine [DATE] and [TIME] to create a unique timestamp.

What is the best way to do this in Logstash?

Kind regards and thanks in advance,
Andreas


(system) #2