Add a field depending on first log

Elastic Stack Logstash
1

Hello,

I would like to add a field in my logstash pipeline. But this field would be a dynamic field.

Here is a sample of my file :

date_log;job_id;service_id;name;progress;status;duration

2021-06-01 16:16:03;nJzP4g;7759;masterJob;begin;;
2021-06-01 16:16:04;nJzP4g;7759;subJob;begin;;
2021-06-01 16:16:05;nJzP4g;7759;subJob;end;success;13
2021-06-01 16:16:06;nJzP4g;7759;masterJob;end;success;62114
2021-06-01 16:16:07;AJzhd2;7760;secondaryJob;begin;;
2021-06-01 16:16:09;AJzhd2;7760;subJob2;begin;;
2021-06-01 16:16:10;AJzhd2;7760;subJob2;end;success;13
2021-06-01 16:16:12;AJzhd2;7760;secondaryJob;end;success;62114

I would like to add "type" field for all jobs that are between the beginning of masterJob and the end :

Example of the final result :

date_log;job_id;service_id;name;progress;status;duration;type

2021-06-01 16:16:03;nJzP4g;7759;masterJob;begin;;;masterJob
2021-06-01 16:16:04;nJzP4g;7759;subJob;begin;;;masterJob
2021-06-01 16:16:05;nJzP4g;7759;subJob;end;success;13;masterJob
2021-06-01 16:16:06;nJzP4g;7759;masterJob;end;success;62114;masterJob
2021-06-01 16:16:07;AJzhd2;7760;secondaryJob;begin;;;secondaryJob
2021-06-01 16:16:09;AJzhd2;7760;subJob2;begin;;;secondaryJob
2021-06-01 16:16:10;AJzhd2;7760;subJob2;end;success;13;secondaryJob
2021-06-01 16:16:12;AJzhd2;7760;secondaryJob;end;success;62114;secondaryJob

So :
if name = masterJob and status = begin
then all the following lines will have type = masterJob until we have the line name = masterJob and status = end

Same thing for "secondaryJob"

How can I do that with the different filters of logstash ?

Thanks in advance.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.