Update field value based on reference variable

Elastic Stack Logstash
1

Using Logstash, I'm parsing a log file that contains a timestamp value every time a new process starts. Each process starts and logs its own run time starting from 0 to the time the process completes. I'm trying to visualize this in Kibana, and would like to be able to update the process time to the real-time timestamp being logged for that process.

Sample Input:

12:00:00 Process A starts: 
     0:00:01 Step 1
     0:00:04 Step 2
     ........
     0:10:01 Process Ends
%Random Log lines of no interest
12:10:05 Process B starts:
     0:00:01 Step 1
     0:00:04 Step 2
     ........
     0:10:01 Process Ends
.........

I'd like to parse this such that each timestamp is updated with the starting timestamp of the process.

Sample Output:

12:00:00 Process A starts: 
     12:00:01 Step 1
     12:00:04 Step 2
     ........
     12:10:01 Process Ends
%Random Log lines of no interest
12:10:05 Process B starts:
     12:10:06 Step 1
     12:10:09 Step 2
     ........
     12:20:06 Process Ends
.........

If this can be done either in Logstash or Elasticsearch that would be really helpful.

2

Hi,

Take a look to the aggregate filter. I think it's a good choice in your case.

Cad.

3

Hi Cad,
I could use the aggregate filter, but it works by aggregating messages with common field i.e task id. Would that mean that my process logs would have to contain a common field to aggregate?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.