# Logstash - Difference between date into new field

**URL:** <https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818>\
**Category:** Logstash\
**Created:** [September 9, 2021, 8:59pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818 "2021-09-09T20:59:41Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![stemons](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stemons/32/84571_2.png) [@stemons](https://discuss.elastic.co/u/stemons)\
**Post date:** [September 9, 2021, 8:59pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818/1 "2021-09-09T20:59:41Z")

</div>

Hello team,  
I'm trying to add a new field to metricbeat data collection through logstash. The idea is to create a field from the difference between two dates (@timestamp and system.process.cpu.start\_date).

I've made some test, but the field created in elastic is empty.

Here is the logstash configuration

```auto
input {
  beats {
    port => 5044
  }
}
filter {
  date {
         match => ["@timestamp", "ISO8601"]
         target => "start_date"
  }
  date {
         match => ["system.process.cpu.start_time", "ISO8601"]
         target => "end_date"
  }
  ruby {
        init => "require 'time'"
        code => "
                 duration = (event.get('start_date') - event.get('end_date')) rescue nil;
                 event.set('system.process.cpu.duration', duration);
                "
  }
}

```

---

<div class="post-metadata">

**Author:** ![Iker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iker/32/91708_2.png) [@Iker](https://discuss.elastic.co/u/Iker)\
**Post date:** [September 9, 2021, 9:13pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818/2 "2021-09-09T21:13:15Z")

</div>

I haven't tried your code but the start date, must be subtracted from the finish date, try with this, haven't tested but should be fine:

```auto
duration = event.get('end_date').to_i - event.get('start_date').to_i 

```

There is also a math filter for logstash if you like to try it.

> **[GitHub - logstash-plugins/logstash-filter-math: This plugin provides the...](https://github.com/logstash-plugins/logstash-filter-math)**
>
> This plugin provides the ability to do various simple math operations (addition, subtraction, multiplication and division) on document fields - GitHub - logstash-plugins/logstash-filter-math: This ...

---

<div class="post-metadata">

**Author:** ![stemons](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stemons/32/84571_2.png) [@stemons](https://discuss.elastic.co/u/stemons)\
**Post date:** [September 9, 2021, 9:23pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818/3 "2021-09-09T21:23:10Z")

</div>

> [@stemons](#):
>
> ```auto
> ruby {
> init => "require 'time'"
> code => "
> duration = (event.get('start_date') - event.get('end_date')) rescue nil;
> event.set('system.process.cpu.duration', duration);
> "
> }
> 
> ```

Using this now I'm getting always 0. What should be the result unit ? days?

system.process.cpu.duration 0  
@timestamp Sep 9, 2021 @ 23:21:08.700  
system.process.cpu.start\_time Sep 9, 2021 @ 23:19:44.000

```auto
date {
         match => ["@timestamp", "ISO8601"]
         target => "end_date"
  }
  date {
         match => ["system.process.cpu.start_time", "ISO8601"]
         target => "start_date"
  }
ruby {
        init => "require 'time'"
        code => "
                 duration = (event.get('end_date').to_i - event.get('start_date').to_i) rescue nil;
                 event.set('system.process.cpu.duration', duration);
                "
  }

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 9, 2021, 9:42pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818/4 "2021-09-09T21:42:08Z")

</div>

> [@stemons](#):
>
> `system.process.cpu.start_time`

logstash does not use the same syntax to refer to objects nested inside objects that filebeat documentation and elasticsearch use. In logstash that field would be referred to as

```
[system][process][cpu][start_time]

```

This allows logstash to unambiguously refer to fields that have . in their name.

---

<div class="post-metadata">

**Author:** ![stemons](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stemons/32/84571_2.png) [@stemons](https://discuss.elastic.co/u/stemons)\
**Post date:** [September 9, 2021, 9:47pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818/5 "2021-09-09T21:47:51Z")

</div>

Thanks. Now I'm getting epoch time format, but only for the field @timestamp.

For the other field I'm getting a wrong value (2021). Seems it's extracting the year from the date.

date\_end\_i  
1631224028

date\_start\_i  
2021

```auto
ruby {
        init => "require 'time'"
        code => "
                 duration = (event.get('@timestamp').to_i - event.get('[process][cpu][start_time]').to_i) rescue nil;
                 date_end_i = event.get('@timestamp').to_i;
                 date_start_i = event.get('[process][cpu][start_time]').to_i;
                 event.set('date_end_i',date_end_i);
                 event.set('date_start_i',date_start_i);
                 event.set('system.process.cpu.duration', duration);
                "
  }

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 9, 2021, 9:54pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818/6 "2021-09-09T21:54:09Z")

</div>

> [@stemons](#):
>
> `date_start_i = event.get('[process][cpu][start_time]').to_i;`

That is going to take a string like "2021-03-29T04:24:52.000Z" and .to\_i will just pull the leading 2021 from it. You need to parse that using a date filter to convert it to a LogStash::Timestamp (that's what a date filter always produces) so that .to\_i returns a number of seconds.

---

<div class="post-metadata">

**Author:** ![stemons](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stemons/32/84571_2.png) [@stemons](https://discuss.elastic.co/u/stemons)\
**Post date:** [September 9, 2021, 10:02pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818/7 "2021-09-09T22:02:17Z")

</div>

> [@stemons](#):
>
> ```auto
> date {
> match => ["system.process.cpu.start_time", "ISO8601"]
> target => "end_date"
> }
> 
> ```

It worked! Thanks a lot

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 7, 2021, 10:02pm UTC](https://discuss.elastic.co/t/logstash-difference-between-date-into-new-field/283818/8 "2021-10-07T22:02:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.