# Sum of derivated data by particular fieald **URL:** \ **Category:** Kibana\ **Created:** [September 11, 2018, 2:02pm UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153 "2018-09-11T14:02:51Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Ludek](https://avatars.discourse-cdn.com/v4/letter/l/90db22/32.png) [@Ludek](https://discuss.elastic.co/u/Ludek)\ **Post date:** [September 11, 2018, 2:02pm UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/1 "2018-09-11T14:02:51Z")
Hi there, I have data from a lot of servers and need to summarize. All data are counting up, so I need derivate by hostname. After that make sum. My sample data Time hostname actions.add header [time] mx1 29 [time] mx2 25 [time] mx3 25 [time] mx4 27 [time] mx1 26 [time] mx2 25 [time] mx3 25 [time] mx4 26 [time] mx1 24 [time] mx2 25 [time] mx3 25 [time] mx4 25 I'd like to do this: Max 'actions.add header' group by hostname -\> derivate -\> sum from all servers Any idea how to do this in Kibana? ---
**Author:** ![Ludek](https://avatars.discourse-cdn.com/v4/letter/l/90db22/32.png) [@Ludek](https://discuss.elastic.co/u/Ludek)\ **Post date:** [September 12, 2018, 11:08am UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/2 "2018-09-12T11:08:06Z")
I thought that, this is it, but doesn't work ![Kibana-graph](https://us1.discourse-cdn.com/elastic/original/3X/d/b/dbcf926d3d32494537fb64b75c55bb656bf8f023.png) ---
**Author:** ![jbudz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jbudz/32/45922_2.png) [@jbudz](https://discuss.elastic.co/u/jbudz)\ **Post date:** [September 13, 2018, 2:40pm UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/3 "2018-09-13T14:40:10Z")
This looks exactly right to me, can you check your developer tools for the response from /metrics/vis/data? If the error is anything related to a null pointer exception I'm thinking it's a bug, [https://github.com/elastic/elasticsearch/issues/27544](https://github.com/elastic/elasticsearch/issues/27544). ---
**Author:** ![Ludek](https://avatars.discourse-cdn.com/v4/letter/l/90db22/32.png) [@Ludek](https://discuss.elastic.co/u/Ludek)\ **Post date:** [September 14, 2018, 12:02pm UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/4 "2018-09-14T12:02:34Z")
When I try to use your example, I have this message only when I have a small number of samples in db. I increase it for test and it was OK. But, when I extend the query more close to what I need (meens derivative function) it returns me the same message: ``` { "error": { "root_cause": [], "type": "search_phase_execution_exception", "reason": "", "phase": "fetch", "grouped": true, "failed_shards": [], "caused_by": { "type": "null_pointer_exception", "reason": null } }, "status": 503 } ``` My query is: ``` GET index-*/_search { "size": 1, "query": { "range": { "date": { "gte": "now-15m/m", "lte": "now" } } }, "aggs": { "timeseries": { "date_histogram": { "field": "date", "interval": "10s" }, "aggs": { "maxRJ": { "max": { "field": "actions.reject" } }, "calculation": { "bucket_script": { "buckets_path": { "maxRJ": "maxRJ" }, "script": { "source": "params.maxRJ" } } }, "derive": { "derivative": { "buckets_path": "calculation" } }, "cumsum": { "cumulative_sum": { "buckets_path": "derive" } } } } } } ``` ---
**Author:** ![jbudz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jbudz/32/45922_2.png) [@jbudz](https://discuss.elastic.co/u/jbudz)\ **Post date:** [September 14, 2018, 2:33pm UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/5 "2018-09-14T14:33:47Z")
Can you share your Elasticsearch version? Is upgrading to 6.4 an option? It looks like there was a fix merged then. ---
**Author:** ![Ludek](https://avatars.discourse-cdn.com/v4/letter/l/90db22/32.png) [@Ludek](https://discuss.elastic.co/u/Ludek)\ **Post date:** [September 17, 2018, 8:58am UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/6 "2018-09-17T08:58:15Z")
I'm on 5.6.11. And for now, it's not possible to upgrade to 6.X. We are planning it in future, but it's about 1 year or more. ---
**Author:** ![jbudz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jbudz/32/45922_2.png) [@jbudz](https://discuss.elastic.co/u/jbudz)\ **Post date:** [September 21, 2018, 3:06am UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/7 "2018-09-21T03:06:50Z")
Okay, as far as I can tell our best course is to plot this outside of TSVB then. Timelion is an option, and will crunch derivative and cumulative sum in the Kibana server avoiding the elasticsearch bug. a query would look something like: ```auto .es(metric=max:actions.add_header, split=hostname:10).derivative().cusum() ``` ---
**Author:** ![Ludek](https://avatars.discourse-cdn.com/v4/letter/l/90db22/32.png) [@Ludek](https://discuss.elastic.co/u/Ludek)\ **Post date:** [October 2, 2018, 12:31pm UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/8 "2018-10-02T12:31:06Z")
Had no time to play with this. But I thought about Timelion before. OK, thanks for help. I'll use Timelion. ---
**Author:** ![Ludek](https://avatars.discourse-cdn.com/v4/letter/l/90db22/32.png) [@Ludek](https://discuss.elastic.co/u/Ludek)\ **Post date:** [October 3, 2018, 12:22pm UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/9 "2018-10-03T12:22:07Z")
I have partial success. `.es(index=spam-status-*, timefield=date, metric="max:actions.add header",split=hostname:20).derivative()` returns this graph ![kibana-timelion1](https://us1.discourse-cdn.com/elastic/original/3X/3/e/3e394648ed2fa0f6ec56b8b3fb594521ae0e63da.png) But when I use cusum() function, it not makes sense what I get for me. `.es(index=spam-status-*, timefield=date, metric="max:actions.add header",split=hostname:20).derivative().cusum()` ![kibana-timelion2](https://us1.discourse-cdn.com/elastic/original/3X/c/e/ce64edb156317b490c2216bab3dc36b99993b2b3.png) I expected the sum of all derivative series. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 31, 2018, 12:22pm UTC](https://discuss.elastic.co/t/sum-of-derivated-data-by-particular-fieald/148153/10 "2018-10-31T12:22:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.