# Substracting event fields in partitioned data

**URL:** <https://discuss.elastic.co/t/substracting-event-fields-in-partitioned-data/249119>\
**Category:** Logstash\
**Created:** [September 18, 2020, 2:32pm UTC](https://discuss.elastic.co/t/substracting-event-fields-in-partitioned-data/249119 "2020-09-18T14:32:48Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![JorritvdLaan](https://avatars.discourse-cdn.com/v4/letter/j/ee59a6/32.png) [@JorritvdLaan](https://discuss.elastic.co/u/JorritvdLaan)\
**Post date:** [September 18, 2020, 2:32pm UTC](https://discuss.elastic.co/t/substracting-event-fields-in-partitioned-data/249119/1 "2020-09-18T14:32:48Z")

</div>

Hello,

I am working with a set of devices that pushes data to my database every 5 minutes. The data contains a field with the device's id and a field with a float value.

Now, for every device I would like to substract the previous value from the current value, and add this value in a new field. I figured I would use the aggregate plugin to substract the values, but I am having trouble sorting the data on the device ids. A complicating factor is that the number of devices varies and can be quite large, so putting a bunch of aggregate filters in parallel is not feasible.

Is it possible to creat such a filter in logstash? Or is logstash not a suitable tool for this problem?

Thanks in advance!

Jorrit

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 18, 2020, 3:12pm UTC](https://discuss.elastic.co/t/substracting-event-fields-in-partitioned-data/249119/2 "2020-09-18T15:12:15Z")

</div>

You might be able to do that using an [update script](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html) in elasticsearch. You would set the document id equal to the device id so that you can reference the previous version of the document.

---

<div class="post-metadata">

**Author:** ![JorritvdLaan](https://avatars.discourse-cdn.com/v4/letter/j/ee59a6/32.png) [@JorritvdLaan](https://discuss.elastic.co/u/JorritvdLaan)\
**Post date:** [September 22, 2020, 1:50pm UTC](https://discuss.elastic.co/t/substracting-event-fields-in-partitioned-data/249119/3 "2020-09-22T13:50:02Z")

</div>

Hi Badger,

Thank you for your reply. I have been playing around with the update functionality you suggested. Unfortunately I could not make it work, since (as far as I could see) the API does not allow the usage of values of other documents, which is needed for calculating the difference between two documents. This is the same issue I encountered using Ingest nodes and Transforms.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 22, 2020, 2:40pm UTC](https://discuss.elastic.co/t/substracting-event-fields-in-partitioned-data/249119/4 "2020-09-22T14:40:32Z")

</div>

If they are different documents then that would not work. As I noted, I was assuming you would set the document id equal to the device id so that you can overwrite/update the document.

You could try fetching the existing document with an elasticsearch filter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 20, 2020, 2:40pm UTC](https://discuss.elastic.co/t/substracting-event-fields-in-partitioned-data/249119/5 "2020-10-20T14:40:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.