Logstash - adding shared field to link related logs


#1

i'm processing logs from multiple voice switches with logstash 5.0.1.

the logs are often related to other logs by "call_ids". i extract these into an array:

event.set("call_ids", event.get("message").scan(/\s+Z\S+\s+(\S+)@\S+/).flatten)

a master record will usually be present and will have a few of these:

LOG1 - call_ids:31967-CJ-0679798b-69dbbb183, 27368-YT-06797982-6342ccc83, 31810-JE-0679797d-69dfcbed0, 76838398-3690563729-694540 ...

most of these call_ids will appear in one other related log which is received within a minute of the others:

LOG2 - call_ids:31967-CJ-0679798b-69dbbb183
LOG3 - call_ids:27368-YT-06797982-6342ccc83
LOG4 - call_ids:31810-JE-0679797d-69dfcbed0

however the master log will often appear after the other device logs.

i would like to add a field to each of these events which links them together (call_id_link).

however, i think i need to make use of global/class variables to do so and i have no experience in ruby.

do you guys have any advice on how to proceed?

note that i tried using the aggregate filter using call_ids as task_ids but could not get this work where the master log appears after the other logs.

thanks for your help,

colm


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.