# A new field with a value from the difference between the two existing fields

**URL:** <https://discuss.elastic.co/t/a-new-field-with-a-value-from-the-difference-between-the-two-existing-fields/243670>\
**Category:** Kibana\
**Created:** [August 4, 2020, 8:19am UTC](https://discuss.elastic.co/t/a-new-field-with-a-value-from-the-difference-between-the-two-existing-fields/243670 "2020-08-04T08:19:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alex9](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex9/32/72742_2.png) [@Alex9](https://discuss.elastic.co/u/Alex9)\
**Post date:** [August 4, 2020, 8:19am UTC](https://discuss.elastic.co/t/a-new-field-with-a-value-from-the-difference-between-the-two-existing-fields/243670/1 "2020-08-04T08:19:27Z")

</div>

Hello.  
Please tell me the direction to the solution.  
There is a service that generates records. The values include: fields with a unique object id, a record of the object's action and time.  
Question: how to get new fields showing the difference in the values of the existing fields from different records with the same id?  
What tools can be used to implement this?  
When creating dashboards? Or maybe by means of Elasticsearch itself?  
Thank you in advance!

 ![Annotation 2020-08-04 111218](https://us1.discourse-cdn.com/elastic/original/3X/2/e/2e633835d47eaad9c1c33b214a8719d74ec75d37.png)

---

<div class="post-metadata">

**Author:** ![mattkime](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattkime/32/43522_2.png) [@mattkime](https://discuss.elastic.co/u/mattkime)\
**Post date:** [August 4, 2020, 3:33pm UTC](https://discuss.elastic.co/t/a-new-field-with-a-value-from-the-difference-between-the-two-existing-fields/243670/2 "2020-08-04T15:33:38Z")

</div>

Hello @Alex9

Which version of the stack are you using? How are you ingesting your data?

Ideally we would have a single document for that transaction id.

---

<div class="post-metadata">

**Author:** ![Alex9](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex9/32/72742_2.png) [@Alex9](https://discuss.elastic.co/u/Alex9)\
**Post date:** [August 10, 2020, 11:16am UTC](https://discuss.elastic.co/t/a-new-field-with-a-value-from-the-difference-between-the-two-existing-fields/243670/3 "2020-08-10T11:16:57Z")

</div>

Hello 🙂  
Elasticsearch version 6.6.1. I receive the data in a script directly into Elasticsearch.  
Is it possible to solve the issue using Elasticsearch + Kibana?

But, I also prepared a cluster of version 7.8.1.  
From the service, data is sent by a script to the / elk / data directory  
Logstash takes the data from the directory and sends it to Elasticsearch.  
Current configuration:

```auto
input {
  file {
    path => "/elk/data/service*"
    start_position => "beginning"
  }
}
filter {
    csv {
	separator => "|"
	    columns => ["object_#", "id", "checkpoint", "time"]
    }
    grok {
        match => { "time" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
    date {
    match => ["timestamp", "dd/MM/YY HH:mm"]
    }
    }
output {
stdout { codec => rubydebug }
    elasticsearch {
        hosts => ["localhost:9200"]
        user => "elastic"
        password => "my_password..."
        cacert => '/usr/share/logstash/elastic-certificates.pem'
        index => "service2"
        }
}

```

The data in the catalog /elk/data looks like this:  
object 1|BzjZ5P|start|12:30:00 PM  
object 1|BzjZ5P|fisnish|12:35:00 PM  
object 2|u827Qp|start|1:05:00 PM  
object 2|u827Qp|fisnish|1:15:00 PM  
object 3|2caUKH|start|1:35:00 PM  
object 3|2caUKH|finish|1:39:00 PM  
object 4|adaf12|start|1:40:00 PM

**Thanks for your answer and time!**

---

<div class="post-metadata">

**Author:** ![mattkime](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattkime/32/43522_2.png) [@mattkime](https://discuss.elastic.co/u/mattkime)\
**Post date:** [August 10, 2020, 12:31pm UTC](https://discuss.elastic.co/t/a-new-field-with-a-value-from-the-difference-between-the-two-existing-fields/243670/4 "2020-08-10T12:31:55Z")

</div>

I'd try using the aggregate filter to get the data you want into a single document. Everything else is much easier this way. [https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 7, 2020, 12:31pm UTC](https://discuss.elastic.co/t/a-new-field-with-a-value-from-the-difference-between-the-two-existing-fields/243670/5 "2020-09-07T12:31:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.