# ELK watcher : painless script - stream.filter

**URL:** <https://discuss.elastic.co/t/elk-watcher-painless-script-stream-filter/228392>\
**Category:** Elasticsearch\
**Created:** [April 16, 2020, 6:30pm UTC](https://discuss.elastic.co/t/elk-watcher-painless-script-stream-filter/228392 "2020-04-16T18:30:43Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jags](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jags/32/45922_2.png) [@Jags](https://discuss.elastic.co/u/Jags)\
**Post date:** [April 16, 2020, 6:30pm UTC](https://discuss.elastic.co/t/elk-watcher-painless-script-stream-filter/228392/1 "2020-04-16T18:30:43Z")

</div>

Below is the reslut of Elastic query with derivative aggregation

"utils\_per\_5m": {  
"buckets": [  
{  
"key\_as\_string": "2020-04-15T21:10:00.000Z",  
"doc\_count": 1,  
"utils": {  
"value": 924  
},  
"key": 1586985000000  
},  
{  
"key\_as\_string": "2020-04-15T21:15:00.000Z",  
"doc\_count": 1,  
"utils": {  
"value": 0  
},  
"utils\_deriv": {  
"value": -924  
},  
"key": 1586985300000  
}  
]  
}

utils\_per\_5m.buckets array has two objects. Since using derivative, only the second object in the array will have util\_deriv .  
Usecase : filter utils\_per\_5m.buckets array (2 objects ) with the condition as  
utils\_per\_5m.buckets.stream().filter(poll -\> poll.utils\_deriv.value != null && poll.utils\_deriv.value \< 0).collect(Collectors.toList())

while executing the script in painless, getting NPE, though null check in pace for poll.utils\_deriv.value NULL prior to poll.utils\_deriv.value \< 0 .

But , poll.utils.value == 0 filter working , utils.value exists in both the utils\_per\_5m.buckets array objects.  
utils\_per\_5m.buckets.stream().filter(poll -\> poll.utils.value == 0).collect(Collectors.toList())

Can you clarify why poll.utils\_deriv.value filter throwing NPE even after checking NULL , also second object in the array has matching data.

---

<div class="post-metadata">

**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\
**Post date:** [April 16, 2020, 10:27pm UTC](https://discuss.elastic.co/t/elk-watcher-painless-script-stream-filter/228392/2 "2020-04-16T22:27:36Z")

</div>

From a quick look, wouldn't you need to check `poll.utils_deriv` for `null` already? If `poll.utils_deriv` is `null` then `poll.utils_deriv.value` will run into a NPE.

PS: Please format your code for better readability 🙂

---

<div class="post-metadata">

**Author:** ![Jags](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jags/32/45922_2.png) [@Jags](https://discuss.elastic.co/u/Jags)\
**Post date:** [April 16, 2020, 11:32pm UTC](https://discuss.elastic.co/t/elk-watcher-painless-script-stream-filter/228392/3 "2020-04-16T23:32:37Z")

</div>

works.. thanks for the insight

---

<div class="post-metadata">

**Author:** ![Jags](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jags/32/45922_2.png) [@Jags](https://discuss.elastic.co/u/Jags)\
**Post date:** [April 17, 2020, 6:40pm UTC](https://discuss.elastic.co/t/elk-watcher-painless-script-stream-filter/228392/4 "2020-04-17T18:40:01Z")

</div>

Here's my result

"aggregations": {  
"name": {  
"buckets": [  
{  
"doc\_count": 8,  
"port": {  
"doc\_count\_error\_upper\_bound": 0,  
"sum\_other\_doc\_count": 0,  
"buckets": [  
{  
"doc\_count": 2,  
"utils\_per\_5m": {  
"buckets": [  
{  
"key\_as\_string": "2020-04-17T18:00:00.000Z",  
"doc\_count": 1,  
"utils": {  
"value": 924  
},  
"key": 1587146400000  
},  
{  
"key\_as\_string": "2020-04-17T18:05:00.000Z",  
"doc\_count": 1,  
"utils": {  
"value": 923  
},  
"utils\_deriv": {  
"value": -1  
},  
"key": 1587146700000  
}  
]  
},  
"key": "Port-1"  
},  
{  
"doc\_count": 2,  
"utils\_per\_5m": {  
"buckets": [  
{  
"key\_as\_string": "2020-04-17T18:00:00.000Z",  
"doc\_count": 1,  
"utils": {  
"value": 876  
},  
"key": 1587146400000  
},  
{  
"key\_as\_string": "2020-04-17T18:05:00.000Z",  
"doc\_count": 1,  
"utils": {  
"value": 870  
},  
"utils\_deriv": {  
"value": -6  
},  
"key": 1587146700000  
}  
]  
},  
"key": "Port-2"  
}  
]  
},  
"key": "Avenal"  
},{ }  
]  
}  
}

painless script :  
POST \_scripts/deviceutil-parser  
{  
"script": {  
"lang": "painless",  
"source": "return ['host\_port\_util\_map': ctx.payload.aggregations.name.buckets.stream().map(p -\> [p.key, p.port.buckets.stream().map(bkts -\> [bkts.key,bkts.utils\_per\_5m.buckets.stream().filter(poll -\> poll.utils.value == 0 && (poll.utils\_deriv != null && poll.utils\_deriv.value \< 0)).collect(Collectors.toList())]).collect(Collectors.toList())]).collect(Collectors.toList())];"  
}  
}

Script output:  
[Avenal, [[Port-1,], [[Port-2,] ]

Question : Since there is mo matching records, getting empty arrays. Is there a better way to filter the steam to return only matching data set ie non empty arrays. Pls ask if you need more info

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 15, 2020, 6:40pm UTC](https://discuss.elastic.co/t/elk-watcher-painless-script-stream-filter/228392/5 "2020-05-15T18:40:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.