# How to calculate and present times between logs

**URL:** <https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324>\
**Category:** Logstash\
**Created:** [August 31, 2022, 7:40am UTC](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324 "2022-08-31T07:40:08Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![amir\_Bialek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amir_bialek/32/109973_2.png) [@amir\_Bialek](https://discuss.elastic.co/u/amir_Bialek)\
**Post date:** [August 31, 2022, 7:40am UTC](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324/1 "2022-08-31T07:40:08Z")

</div>

Hey, new user of elk with Logstash.

I am using this as a confid:

```auto
input {
    file {
        path => "/myapp/Logs/*.slog"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}

filter {
    json{
        source => "message"
    }
}

output {
    elasticsearch {
        hosts => ["elasticsearch:9200"]
        index => "myapp-json-logs"
    }
}

```

And the logs that it read have this format:

```auto
{"@time":"2022-08-22T17:06:24.8070049+03:00" "@Moving":"< important app 1bla 1bla 1bla "}
<other logs..>
<other logs..>
<other logs..>
{"@time":"2022-08-22T17:16:25.9126576+03:00" "@Moving":"> important app bla2 bla2 bla2 "}
<other logs..>
<other logs..>
{"@time":"2022-08-22T17:23:24.8070049+03:00" "@Moving":"< important app 1bla 1bla 1bla "}
<other logs..>
<other logs..>
<other logs..>
{"@time":"2022-08-22T17:26:25.9126576+03:00" "@Moving":"> important app bla2 bla2 bla2" }

```

I am trying to find a way to calculate the time that pass between the log that include `"@Moving":"< important app ` to the log that include `"@Moving":"> important app"`

This value repeats several times in the files.slog

If someone can give me direction I will be very grateful.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 31, 2022, 9:32pm UTC](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324/2 "2022-08-31T21:32:37Z")

</div>

You could use an aggregate filter to do that

```
    if [message] =~ /"@Moving":"/ {
        grok { match => { "message" => '{"@time":"(?<[@metadata][timestamp]>[^"]+)" "@Moving":"(?<[@metadata][msg]>[^"]+)"' } }
        mutate { add_field => { "[@metadata][json]" => '{ "@timestamp": "%{[@metadata][timestamp]}" }' } }
        json { source => "[@metadata][json]" }

        mutate { add_field => { "[@metadata][taskId]" => "anyConstant" } }
        if [@metadata][msg] =~ /^</ {
            aggregate {
                task_id => "%{[@metadata][taskId]}"
                code => 'map["startTime"] = event.get("@timestamp").to_f'
                map_action => "create"
            }
        } else {
            aggregate {
                task_id => "%{[@metadata][taskId]}"
                code => 'event.set("delta", event.get("@timestamp").to_f - map["startTime"])'
                map_action => "update"
                end_of_task => true
            }
        }
    }

```

which will produce

```
     "delta" => 181.1056525707245,

```

for the second pair of lines. Note that using aggregate requires pipeline.workers to be 1 and pipeline.ordered to be true.

---

<div class="post-metadata">

**Author:** ![amir\_Bialek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amir_bialek/32/109973_2.png) [@amir\_Bialek](https://discuss.elastic.co/u/amir_Bialek)\
**Post date:** [September 1, 2022, 1:39pm UTC](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324/3 "2022-09-01T13:39:54Z")

</div>

Hey @ [Badger](https://discuss.elastic.co/u/Badger), thx for the help.  
I just applied this setting in the .config, the result was:

 ![Untitled](https://us1.discourse-cdn.com/elastic/original/3X/5/8/5852bb30a1f5a1e32bfbef7d5fe7f5d8942aa52e.png)  
(it only discover one log now: `Sep 1, 2022 @ 15:25:47.491	_@timestamp:%{[@metadata][timestamp]} @Moving:< important app @time:Aug 22, 2022 @ 16:06:24.807 @timestamp:Sep 1, 2022 @ 15:25:47.491 @version:1 host:b4b8f10395e3 message:{"@time":"2022-08-22T17:06:24.8070049+03:00", "@Moving":"< important app "} path:/XwinSys/Logs/log.slog tags:_grokparsefailure, _timestampparsefailure _id:L446-YIBF2xDdXQmqk8h _index:xwinsys-json-logs _score: - _type:_doc` )

I do not get the "delta" field..

My log file:

```auto
{"@time":"2022-08-22T17:06:24.8070049+03:00", "@Moving":"< important app "}
{"@time":"2022-08-22T17:16:25.9126576+03:00", "@Moving":"> important app "}

```

.config:

```auto
input {
    file {
        path => "Logs/*.slog"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}

filter {
    json{
        source => "message"
    }

    if [message] =~ /"@Moving":"/ {
        grok { match => { "message" => '{"@time":"(?<[@metadata][timestamp]>[^"]+)" "@Moving":"(?<[@metadata][msg]>[^"]+)"' } }
        mutate { add_field => { "[@metadata][json]" => '{ "@timestamp": "%{[@metadata][timestamp]}" }' } }
        json { source => "[@metadata][json]" }

        mutate { add_field => { "[@metadata][taskId]" => "anyConstant" } }
        if [@metadata][msg] =~ /^</ {
            aggregate {
                task_id => "%{[@metadata][taskId]}"
                code => 'map["startTime"] = event.get("@timestamp").to_f'
                map_action => "create"
            }
        } else {
            aggregate {
                task_id => "%{[@metadata][taskId]}"
                code => 'event.set("delta", event.get("@timestamp").to_f - map["startTime"])'
                map_action => "update"
                end_of_task => true
            }
        }
    }
}

output {
    elasticsearch {
        hosts => ["elasticsearch:9200"]
        index => "json-logs"
    }
}

```

?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 1, 2022, 2:04pm UTC](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324/4 "2022-09-01T14:04:02Z")

</div>

You originally said your log format was

> [@amir\_Bialek](#):
>
> ```auto
> {"@time":"2022-08-22T17:23:24.8070049+03:00" "@Moving":"< important app 1bla 1bla 1bla "}
> <other logs..>
> 
> ```

now you are saying it is

> [@](#):
>
> ```auto
> {"@time":"2022-08-22T17:06:24.8070049+03:00", "@Moving":"< important app "}
> 
> ```

You will need to make appropriate changes to the configuration for the different format.

---

<div class="post-metadata">

**Author:** ![amir\_Bialek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amir_bialek/32/109973_2.png) [@amir\_Bialek](https://discuss.elastic.co/u/amir_Bialek)\
**Post date:** [September 3, 2022, 10:06am UTC](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324/6 "2022-09-03T10:06:59Z")

</div>

@ [Badger](https://discuss.elastic.co/u/Badger) many thanks for your help - I think I understand the code now! :))

Pls ignore my other message. and again - thank you ! : ))

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 1, 2022, 10:07am UTC](https://discuss.elastic.co/t/how-to-calculate-and-present-times-between-logs/313324/7 "2022-10-01T10:07:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.