# Transform free bytes into GB in ES watch for hard disk utilisation

**URL:** <https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-alerting\
**Created:** [August 6, 2018, 12:01pm UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162 "2018-08-06T12:01:46Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![sahil\_sawhney](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sahil_sawhney/32/88932_2.png) [@sahil\_sawhney](https://discuss.elastic.co/u/sahil_sawhney)\
**Post date:** [August 6, 2018, 12:01pm UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/1 "2018-08-06T12:01:46Z")

</div>

In my scenario i want to simply transform free bytes of Hard disk into GB(wrt to .monitoring-es-6\* index). Doing this operation in groovy was a bit easy. But wrt to v5.6 of ES i need to use java 8 to do so.  
Applieng map transforms the value but it do not return me back the object on which i was working on.  
my watcher json is something like this

```
{
  "trigger": {
    "schedule": {
      "interval": "1m"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": [
          ".monitoring-es-6-*"
        ],
        "body": {
          "size" : 0,
          "query": {
            "bool": {
              "filter": {
                "range": {
                  "timestamp": {
                    "gte": "now-2m",
                    "lte": "now"
                  }
                }
              }
            }
          },
          "aggs": {
            "minutes": {
              "date_histogram": {
                "field": "timestamp",
                "interval": "1m"
              },
              "aggs": {
                "nodes": {
                  "terms": {
                    "field": "source_node.ip",
                    "order": {
                      "hdd": "desc"
                    }
                  },
                  "aggs": {
                    "hdd": {
                      "avg": {
                        "field": "node_stats.fs.total.free_in_bytes"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": "if (ctx.payload.aggregations.minutes.buckets.size() == 0) return false; def latest = ctx.payload.aggregations.minutes.buckets[-1]; def node = latest.nodes.buckets[0]; return node.hdd.value <= 7000000000000L;"
  },
  "actions": {
    "send_email": { 
      "transform": {
        "script": "def latest = ctx.payload.aggregations.minutes.buckets[-1]; return latest.nodes.buckets.stream().filter(it -> it.hdd.value <= 7000000000000L).collect(Collectors.toList());"
      },
      "email": {
        "to": "sahil.sawhney@knoldus.in", 
        "subject": "Watcher Notification - LOW FREE HARD DISK ON SATURN ES",
        "body": "Nodes with LOW FREE HARD DISK (BELOW 100):\n\n{{#ctx.payload._value}}\"{{key}}\" - FREE HARD DISK is {{hdd.value}}%\n{{/ctx.payload._value}}"
      }
    }
  }
}

```

Can anyone help me with the logic to implement this transformation?

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 6, 2018, 12:56pm UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/2 "2018-08-06T12:56:30Z")

</div>

hey,

how about sharing your groovy script and we transform it into painless?

--Alex

---

<div class="post-metadata">

**Author:** ![sahil\_sawhney](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sahil_sawhney/32/88932_2.png) [@sahil\_sawhney](https://discuss.elastic.co/u/sahil_sawhney)\
**Post date:** [August 6, 2018, 1:07pm UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/3 "2018-08-06T13:07:39Z")

</div>

hey @spinscale  
following is my groovy script wrt es 2.4

`def latest = ctx.payload.aggregations.minutes.buckets[-1]; return latest.nodes.buckets.findAll ({ return it.available_disk && it.available_disk.value >= 0 }).findAll({it.available_disk.percent=Math.round((it.available_disk.value/it.total_disk.value)*100); it.available_disk.percent < 15 })`

i want to transform for it.available\_disk.percent

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 6, 2018, 2:24pm UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/4 "2018-08-06T14:24:42Z")

</div>

hey.

ok, so you have to convert your groovy closures into java lambdas (which painless reuses).

Something like this to give you a first hint

```auto
def last = ctx.payload.aggregations.foo.buckets.get(ctx.payload.aggregations.foo.buckets.size()-1);
def hosts = last.nodes.buckets.stream()
  .filter(b -> b.available_disk != null && b.available_disk.value > 0)
  .filter(b -> Math.round(b.available_disk.value/b.total_disk.value)*100 < 100)
  .map(b -> b.key)
  .collect(Collectors.toList());
return ['hosts': hosts]

```

---

<div class="post-metadata">

**Author:** ![sahil\_sawhney](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sahil_sawhney/32/88932_2.png) [@sahil\_sawhney](https://discuss.elastic.co/u/sahil_sawhney)\
**Post date:** [August 6, 2018, 7:08pm UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/5 "2018-08-06T19:08:05Z")

</div>

Hey @spinscale  
i have made certain amendments to the snippet you sent

> ```
> def latest = ctx.payload.aggregations.minutes.buckets.get(ctx.payload.aggregations.minutes.buckets.size() - 1);
> def result = latest.nodes.buckets.stream().filter(it -> it != null && it.free_disk != null && it.free_disk.value > 0).filter(it -> {float free_percent = (float) it.free_disk.value / it.total_disk.value * 100; return free_percent < 100;}).map(it -> it.key).collect(Collectors.toList());
> return result; 
> 
> ```

Now the problem is that the map statement transforms the payload and leave only the ip address of the host in the payload.  
But i want to

1. Introduce new filed in payload, keeping the original ones(want to introduce the field free\_percent)
2. transform the field available\_in\_bytes from byte count to GB count.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 7, 2018, 6:24am UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/6 "2018-08-07T06:24:47Z")

</div>

just return the original payload and **add** a field.

```auto
"script" "def payload = ctx.payload; payload.hosts = ... ; return payload"

```

---

<div class="post-metadata">

**Author:** ![sahil\_sawhney](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sahil_sawhney/32/88932_2.png) [@sahil\_sawhney](https://discuss.elastic.co/u/sahil_sawhney)\
**Post date:** [August 8, 2018, 8:44am UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/7 "2018-08-08T08:44:20Z")

</div>

@spinscale Got a way to transform the payload fields. We have to do something like

```
def latest = ctx.payload.aggregations.minutes.buckets.get(ctx.payload.aggregations.minutes.buckets.size() - 1);
def result = latest.nodes.buckets.stream().filter(it -> it != null && it.free_disk != null && it.free_disk.value > 0).filter(it -> {it.free_disk.value = it.free_disk.value / 1000000000;it.total_disk.value = it.total_disk.value / 1000000000; return (float) it.free_disk.value / it.total_disk.value * 100 < 100;}).collect(Collectors.toList());
return result;
```

---

<div class="post-metadata">

**Author:** ![sahil\_sawhney](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sahil_sawhney/32/88932_2.png) [@sahil\_sawhney](https://discuss.elastic.co/u/sahil_sawhney)\
**Post date:** [August 8, 2018, 9:31am UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/8 "2018-08-08T09:31:31Z")

</div>

So below is an example which shows how field can be transformed as well as added into payload

```
def latest = ctx.payload.aggregations.minutes.buckets.get(ctx.payload.aggregations.minutes.buckets.size() - 1);
def result = latest.nodes.buckets.stream().filter(it -> it != null && it.free_disk != null && it.free_disk.value > 0).filter(it -> {def res = (float) it.free_disk.value / it.total_disk.value * 100; it.free_disk.percent = Math.round(res * 100.0) / 100.0; it.free_disk.value = Math.round((it.free_disk.value / 1000000000) * 100.0) / 100.0; it.total_disk.value = Math.round((it.total_disk.value / 1000000000) * 100.0) / 100.0; return res < 100;}).collect(Collectors.toList());
return result;

```

By this i would conclude that painless is something which merges functionality provided by java 8 and groovy(as per 2.4).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 5, 2018, 9:31am UTC](https://discuss.elastic.co/t/transform-free-bytes-into-gb-in-es-watch-for-hard-disk-utilisation/143162/9 "2018-09-05T09:31:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.