# Safe numerical return value in scripted fields. Avoiding "null pointers" in filters using scripted fields

**URL:** <https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816>\
**Category:** Kibana\
**Tags:** painless\
**Created:** [June 7, 2019, 9:58pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816 "2019-06-07T21:58:10Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![ebalmeida](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebalmeida/32/47103_2.png) [@ebalmeida](https://discuss.elastic.co/u/ebalmeida)\
**Post date:** [June 7, 2019, 9:58pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/1 "2019-06-07T21:58:10Z")

</div>

Hi,

TL/DR: **Is there a safe return value, for a numeric scripted field, that would behave like a non-existing field in a document (for filtering purposes)?**

Btw, I think this ends up relating to another topic I [posted](https://discuss.elastic.co/t/getting-a-no-viable-alt-exception-only-when-trying-to-filter-using-a-scripted-field/183145/3).

For instance, with the script:

```
def audienciaMilis = doc['data_audiencia_pendente'].value.getMillis();
if (audienciaMilis == null || audienciaMilis == 0)
  return null;
ZoneId timeZone = ZoneId.of(ZoneId.SHORT_IDS.get('BET'));
LocalDate dataAudiencia = LocalDateTime.ofInstant(Instant.ofEpochMilli(audienciaMilis),timeZone).toLocalDate().withDayOfMonth(1);
LocalDate dataNow = Instant.ofEpochMilli(new Date().getTime()).atZone(timeZone).toLocalDate().withDayOfMonth(1);
return ChronoUnit.MONTHS.between(dataNow,dataAudiencia);

```

If I filter it like this (taking the DSL version of a filter created on the dashboard -- this is commented while the version above isn't):

```
{
  "script": {
    "script": {
      "inline": "boolean gte(Supplier s, def v) {return s.get() >= v} boolean lt(Supplier s, def v) {return s.get() < v}gte(() -> { // Busca a data de registro do processo. Nunca deveria ser nula ou zero, mas se for retorna nulo.\ndef audienciaMilis = doc['data_audiencia_pendente'].value.getMillis();\nif (audienciaMilis == null || audienciaMilis == 0)\n return null;\n\n// Instancia uma timezone com o horário de Brasilia e duas datas, uma para o início do mẽs da audiência\n// e uma para o início do mês corrente\nZoneId timeZone = ZoneId.of(ZoneId.SHORT_IDS.get('BET'));\n\nLocalDate dataAudiencia = LocalDateTime.ofInstant(Instant.ofEpochMilli(audienciaMilis),timeZone).toLocalDate().withDayOfMonth(1);\nLocalDate dataNow = Instant.ofEpochMilli(new Date().getTime()).atZone(timeZone).toLocalDate().withDayOfMonth(1);\n\nreturn ChronoUnit.MONTHS.between(dataNow,dataAudiencia);\n }, params.gte) && lt(() -> { // Busca a data de registro do processo. Nunca deveria ser nula ou zero, mas se for retorna nulo.\ndef audienciaMilis = doc['data_audiencia_pendente'].value.getMillis();\nif (audienciaMilis == null || audienciaMilis == 0)\n return null;\n\n// Instancia uma timezone com o horário de Brasilia e duas datas, uma para o início do mẽs da audiência\n// e uma para o início do mês corrente\nZoneId timeZone = ZoneId.of(ZoneId.SHORT_IDS.get('BET'));\n\nLocalDate dataAudiencia = LocalDateTime.ofInstant(Instant.ofEpochMilli(audienciaMilis),timeZone).toLocalDate().withDayOfMonth(1);\nLocalDate dataNow = Instant.ofEpochMilli(new Date().getTime()).atZone(timeZone).toLocalDate().withDayOfMonth(1);\n\nreturn ChronoUnit.MONTHS.between(dataNow,dataAudiencia);\n }, params.lt)",
      "params": {
        "gte": -2,
        "lt": 2,
        "value": ">=-2 <2"
      },
      "lang": "painless"
    }
  }
}

```

It will throw a null pointer exception and I get why. There are documents which do not have a "data\_audiencia\_pendente" so they'll return null, which on turn can't be compared to values.

So, going back to the question, what would be a safe way to do this that allows filters to work even with non-existent values (they do for docs with non-existing fields which are not scripted). In this case I cannot assign a negative or zero value because those are actually valid responses from the script.

Thank you very much for any kind of insight here.

---

<div class="post-metadata">

**Author:** ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)\
**Post date:** [June 14, 2019, 2:22pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/2 "2019-06-14T14:22:09Z")

</div>

Hi Erick,

I think you want to check if your value exists first and only return a value for that case something like this;

```auto
if ( doc['system.cpu.nice.pct'].size() > 0 ) {
    return (doc['system.cpu.nice.pct'].value * 2)
}

```

I got the tip to check `.size()` from an error message returned by Elasticsearch. In your case you might need to check if the size if each of the fields you're referencing in your calculations are greater than zero.

Regards,  
Lee

---

<div class="post-metadata">

**Author:** ![ebalmeida](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebalmeida/32/47103_2.png) [@ebalmeida](https://discuss.elastic.co/u/ebalmeida)\
**Post date:** [June 24, 2019, 1:57pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/3 "2019-06-24T13:57:46Z")

</div>

Hi, thanks. We've switched from checking nulls to checking for size. Thank you very much.

Still it doesn't really solve the problem of actually wanting to return a null value to a scripted field and not having filters freak out with that. Any possible solutions there?

---

<div class="post-metadata">

**Author:** ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)\
**Post date:** [June 24, 2019, 3:55pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/4 "2019-06-24T15:55:35Z")

</div>

Can you just add another filter for "exists"?

```auto
{
  "exists": {
    "field": "your field here"
  }
}

```

---

<div class="post-metadata">

**Author:** ![ebalmeida](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebalmeida/32/47103_2.png) [@ebalmeida](https://discuss.elastic.co/u/ebalmeida)\
**Post date:** [June 24, 2019, 4:27pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/5 "2019-06-24T16:27:11Z")

</div>

I'm afraid that doesn't really work. Not really sure why since it doesn't even throw an error.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/3/e3e32ece86f0b1e6329691e49c894b698ddec426.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/a/6a5303269bd96cfdd916bb77ac5abdd7b82da54d.png)

---

<div class="post-metadata">

**Author:** ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)\
**Post date:** [June 24, 2019, 7:13pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/6 "2019-06-24T19:13:45Z")

</div>

That is very strange. Can you go to the Inspect menu on Discover and see the Request sent to Elasticsearch? It should have a section like this;

```auto
  "query": {
    "bool": {
      "must": [
        {
          "exists": {
            "field": "log.file.path"
          }
        },

```

---

<div class="post-metadata">

**Author:** ![matw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matw/32/13913_2.png) [@matw](https://discuss.elastic.co/u/matw)\
**Post date:** [June 25, 2019, 7:14am UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/7 "2019-06-25T07:14:25Z")

</div>

A word about preventing the null pointer exception, you should check if the doc contains the key:

```
doc.containsKey('data_audiencia_pendente')

```

**data\_audiencia\_pendente** isn't a scripted field, right?  
**cc\_meses\_para\_audiencia\_pendente** is, right?

---

<div class="post-metadata">

**Author:** ![ebalmeida](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebalmeida/32/47103_2.png) [@ebalmeida](https://discuss.elastic.co/u/ebalmeida)\
**Post date:** [July 19, 2019, 2:39pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/8 "2019-07-19T14:39:15Z")

</div>

Sorry for the delay. Got bogged down by the switch from 6.x to 7.x  
Apparently, it does create the request correctly:

```
"query": {
    "bool": {
      "must": [
        {
          "exists": {
            "field": "cc_meses_para_audiencia_pendente"
          }
        },

```

And here's the current incarnation of that scripted field btw:

```
if(doc['data_audiencia_pendente'].size() <= 0)
  return null;
def audienciaMilis = doc['data_audiencia_pendente'].value.getMillis();
ZoneId timeZone = ZoneId.of(ZoneId.SHORT_IDS.get('BET'));
LocalDate dataAudiencia = LocalDateTime.ofInstant(Instant.ofEpochMilli(audienciaMilis),timeZone).toLocalDate().withDayOfMonth(1);
LocalDate dataNow = Instant.ofEpochMilli(new Date().getTime()).atZone(timeZone).toLocalDate().withDayOfMonth(1);
return ChronoUnit.MONTHS.between(dataNow,dataAudiencia);

```

It did change behavior however: Now it simply does nothing when I check for existence or try to limit it.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/3/134c63c324f6ca6dabc0a82dc81bfe8463e4f75e.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/1/51de5e897f99fb4687187896656e760989335e15.png)

---

<div class="post-metadata">

**Author:** ![ebalmeida](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebalmeida/32/47103_2.png) [@ebalmeida](https://discuss.elastic.co/u/ebalmeida)\
**Post date:** [July 19, 2019, 2:40pm UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/9 "2019-07-19T14:40:58Z")

</div>

That's right, data\_audiencia\_pendente is a regular ingested date field. What's the difference between checking for the key or checking for the size?  
We just updated all our hundreds of scripts to check for size instead of checking for null because of Elastic 7.x

---

<div class="post-metadata">

**Author:** ![matw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matw/32/13913_2.png) [@matw](https://discuss.elastic.co/u/matw)\
**Post date:** [July 24, 2019, 7:02am UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/10 "2019-07-24T07:02:08Z")

</div>

Checking for size is another way to do it. so it's fine

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 21, 2019, 7:02am UTC](https://discuss.elastic.co/t/safe-numerical-return-value-in-scripted-fields-avoiding-null-pointers-in-filters-using-scripted-fields/184816/11 "2019-08-21T07:02:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.