Tabular form or report based on logs

Elastic Stack Elasticsearch
1

I have the following logs -

20:00:00 Started processing
20:05:00 Successfully finished
20:05:03 Output file - /temp/file2
20:05:03 Started processing
20:10:10 Successfully finished
20:10:14 Output file - /temp/file34
20:10:15 Started processing
20:15:00 Successfully finished
20:15:03 Output file - /temp/file16

And, I need a report of the form

Start Finish Output file
20:00:00 20:05:00 /temp/file2
20:05:03 20:10:10 /temp/file34
20:10:15 20:15:00 /temp/file16

Is there a way to generate in ESS or kibana using the KQL or ESQL ? Appreciate any idea or help

2

Hello @dsrini-open

Welcome to the community!!

I understand your requirement but currently you are following record by record approach considering every 3 records, right? How do we know if it is for the same record , is there any common id which will distinguish the start-end-outputfile from others?

Example in your record i use unique id

id=1 20:00:00 Started processing
id=1 20:05:00 Successfully finished
id=1 20:05:03 Output file - /temp/file2

than

id=2 20:05:03 Started processing
id=2 20:10:10 Successfully finished
id=2 20:10:14 Output file - /temp/file34

Incase you have any common id like used in above records than Using ES|QL :

FROM 02aug-req
| STATS
    start_time = MIN(CASE(message LIKE "*Started*", time)),
    end_time = MIN(CASE(message LIKE "*finished*", time)),
    output_file = MIN(CASE(message LIKE "*file*", message))
    BY id
| WHERE start_time IS NOT NULL AND end_time IS NOT NULL AND output_file IS NOT NULL
| KEEP start_time, end_time, output_file
| SORT start_time ASC

image
image1104×207 13.3 KB

Thanks!!

3

Wow, Thanks. We are not currently logging the ID. Let me see if that can be done so that the stats are easier to obtain. Appreciate your help.

1 Like