Hi
I want to use aggregate filter to find dynamic task-id
https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html
Here is the scenario I have log like below need to extract "Send&Receive duration" and "send that has not respond".
this is send
2021-07-15 00:00:01,800 INFO CUST.InAB-ServerApp-1234567 [MyService] Packet Processed: A[50] B[0000211]
this is send
2021-07-15 00:00:01,893 INFO ABCD.DaQW-ParityGQQ-1231234 [MyService] Packet Processed: A[60] B[0000465]
this is send
2021-07-15 00:00:01,894 INFO MNBV.ZaQW-ChatCXZ-1478523 [MyService] Packet Processed: A[70] B[0000369]
this is recieve
2021-07-15 00:00:11,719 INFO CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567 [MyService] Normal Packet Received: A[55] B[0000211]
this is recieve
2021-07-15 00:00:11,720 INFO EFGH.GaXZ-Carry2-3456789_ABCD.DaQW-ParityGQQ-1231234 [MyService] Normal Packet Received: A[65] B[0000456]
here is what happen:
step1: find send id
CUST.InAB-ServerApp-1234567
ABCD.DaQW-ParityGQQ-1231234
MNBV.ZaQW-ChatCXZ-1478523
step2: find response id
CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567
EFGH.GaXZ-Carry2-3456789_ABCD.DaQW-ParityGQQ-1231234
FYI: related events structure like this:
Send: CUST.InAB-ServerApp-1234567
Recieve: CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567
step3: check this condition A+5 AND B=B to match related send receive.
2021-07-15 00:00:01,800 INFO CUST.InAB-ServerApp-1234567 [MyService] Packet Processed: A[50] B[0000211]
2021-07-15 00:00:11,719 INFO CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567 [MyService] Normal Packet Received: A[55] B[0000211]
step4: finally show id that not have receive, and duration of each send&receive
expected Output:
id status
MNBV.ZaQW-ChatCXZ-1478523 no receive
CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567 9,919
EFGH.GaXZ-Carry2-3456789_ABCD.DaQW-ParityGQQ-1231234 9,826
Any idea?
Thanks