Track and show transactions progress based on codes

[Murali_Y]

Hello,
We are writing our business flow related transactions start and end state using different state codes i.e. When we pick the transaction it will logged with code 10001 (in progress) and end of it will be logged with 10002 or 100003 (success and failure respectively).

When I plot a table for in progress (state code 10001) transactions table forever shows the same transactions as log entry with 10001 is already made and cannot be overwritten.
How can we dynamically take off the already processed transactions from the table?

Sample log entries:

@timestamp:Sep 10, 2022 @ 20:11:42.677 fingerprint:2222-605819443b6 Item_Name:069931601477 StateCode 10001 jobId:4444-ff-4555 level:Info levelOrdinal:2 logF_BusinessProcessName:Test logType:User machineId:111 machineName:hostname1 message:test | ProcessCode : 10000 rawMessage:{"message":"test| ProcessCode : 10000","level":"Information","logType":"User","timeStamp":"2022-09-10T20:11:42.6778335+05:30"

@timestamp:Sep 10, 2022 @ 20:11:46.531 fingerprint:2322-44444444 Item_Name:069931601477 StateCode 10002 jobId:4444-ff-4555 level:Info levelOrdinal:2 logF_BusinessProcessName:Test logType:User machineId:111 machineName:hostname1 message:test | ProcessCode : 10000 rawMessage:{"message":"test| ProcessCode : 10000","level":"Information","logType":"User","timeStamp":"2022-09-10T20:11:46.531895+05:30"

Please suggest.

[warkolm (Mark Walkom)]

Is the jobId the unique identifier?

[Murali_Y]

@warkolm Thanks for responding. Item_Name field is our unique identifier on which I want to track and show their progress in a table.

[warkolm (Mark Walkom)]

Ok use a terms aggregation on the Item_name and then a top_hits one on the @timestamp to only show the latest event. That way you keep the history for each item for other analytics, but are able to grab the latest one.