Read logfile once, insert two events to elastic

[Tortoise]

Hello @nilsen

As i am not sure about the exact end requirement , looking at a similar usecase if it can be helpful to you :

Tabular form or report based on logs

I have the following logs - 20:00:00 Started processing 20:05:00 Successfully finished 20:05:03 Output file - /temp/file2 20:05:03 Started processing 20:10:10 Successfully finished 20:10:14 Output file - /temp/file34 20:10:15 Started processing 20:15:00 Successfully finished 20:15:03 Output file - /temp/file16 And, I need a report of the form Start Finish Output file 20:00:00 20:05:00 /temp/file2 20:05:03 20:10:10 /temp/file34 20:10:15 20:15:00 /t…

So if i see your logs & if this is available in elastic:

{ logTime: "20/08/25T12:00:00", id: "random-id-01", message: "user1 connected" }
{ logTime: "20/08/25T12:00:01", id: "random-id-01", message: "uploaded file.xml" }
{ logTime: "20/08/25T12:00:02", id: "random-id-01", message: "disconnected" }

Using ES|QL we can get the aggregated record :

FROM test-message
| EVAL 
    user = REPLACE(CASE(message LIKE "* connected*", message, null), " connected", ""),
    filename = REPLACE(CASE(message LIKE "*uploaded*", message, null), "uploaded ", ""),
    action = CASE(message LIKE "*uploaded*", "upload", null),
    disconnect_time = CASE(message LIKE "*disconnected*", logTime, null)
| STATS 
    logTime = MAX(disconnect_time),
    user = MAX(user),
    action = MAX(action),
    filename = MAX(filename)
    BY id
| WHERE logTime IS NOT NULL AND user IS NOT NULL AND action IS NOT NULL AND filename IS NOT NULL
| KEEP logTime, id, user, action, filename
| SORT logTime ASC

Thanks!!