Logstash conf multiline management


#1

Hi,

I have a log file in input to logstash, something like that :
[....]
2015-04-08 00:21:21,248 [11 ] INFO Traitement de l'archive 20150313.tar-gz
2015-04-08 00:21:21,248 [11 ] INFO Extraction de l'archive 20150313.tar-gz
2015-04-08 00:21:57,513 [11 ] INFO Fin de l'extraction de l'archive
[....]

I already have an elapsed filter to calculate the processing time between the "Traitement" line and the "Fin de l'archive" one.
But I'd also like to index with the last line the name of the tar-gz file (here "20150313"). The issue is I need to get it from previous line.

Any help will be appreciated. Thanks


(Rafał Trójniak) #2

Hello Tibo,

Your problem was quite interesting, so doing some research work I used it.

The problem is : When the end event is processed, the beginning event processing is already finished. There is no direct way to access it.

The only idea on how to walk-around that from my POV is :

  • Simply add this information to final event on event generator side (application/script)
  • Use Multiline and connect both events into one.
    The last way Proof Of Concept I had implemented on my githab tibo_case branch. If you would like to still have the separate events, you would have to duplicate the events.

Rules :


Doc :

Hope that helps

Regards,


#3

Thanks a lot, exactly what I needed.

Regards


(system) #4