Updating dates in Painless

[ffknob (Flávio Knob)]

Hi there

today I was a little bit confused with the documentation about the Painless language. My goal was to use the _update_by_query API to correct the year of some date fields. I was happy to read that...

Date fields are exposed as ZonedDateTime, so they support methods like getYear, getDayOfWeek

https://www.elastic.co/guide/en/elasticsearch/painless/7.5/painless-walkthrough.html#modules-scripting-painless-dates

SO I was hoping that the below code would work...

POST my-index/_update_by_query
{
  "script": {
    "source": """
ctx._source.myDate = ctx._source.myDate.withYear(year);
    """,
    "lang": "painless"
  }
}

...but it failed with:

dynamic method [java.lang.String, withYear/1] not found

I managed to get my update to work by first parsing the field, as shown below:

POST my-index/_update_by_query
{
  "script": {
    "source": """
ctx._source.myDate = ZonedDateTime.parse(ctx._source.myDate).withYear(year);
    """,
    "lang": "painless"
  }
}

Was that expected to work this way? Did I misunderstood something?

[oranieri (Octavio Ranieri)]

Hi Flavio,

It depends of the painless context where the script is being executed.

For example, when getting values from source in _update_by_query context (using ctx._source or ctx['_source']), you will receive the value(s) from JSON schema, which doesn't contain a type date.

Let's create a index with dynamic mapping (types will be date, long, boolean respectively, you may get the index mapping to double check):

PUT myindex/_doc/1
{
  "date": "2020-01-01",
  "number": 123,
  "boolean": true
}

On _update_by_query context, it will read the source into a Map with JSON structure:

ctx['_source'] (Map)
      Contains extracted JSON in a Map and List structure for the fields existing in a stored document.

A simple script for checking the types of each field and storing into a debug field.

POST myindex/_update_by_query
{
  "script": {
    "source": """
      ctx._source.debug = "";
      if(ctx._source.date instanceof String)
        ctx._source.debug += '{date} field is java.lang.String.   ';
      if(ctx._source.number instanceof Integer)
        ctx._source.debug += '{number} field is java.lang.Integer.   ';
      if(ctx._source.boolean instanceof Boolean)
        ctx._source.debug += '{boolean} field is java.lang.Boolean.';
    """,
    "lang": "painless"
  }
}

You will notice that the date field is parsed as String and not Date.

...
{
"date" : "2020-01-01",
"number" : 123,
"boolean" : true,
"debug" : "{date} field is java.lang.String.   
           {number} field is java.lang.Integer.   
           {boolean} field is java.lang.Boolean."
}
...

However using painless in the same index in a field context can be manipulated directly as a date:

GET myindex/_search
{
  "script_fields": {
    "my_year_plus_one": {
      "script": {
        "source": "doc['date'].value.year + 1"
      }
    }
  }
}

Result:

...
"fields" : {
  "my_year_plus_one" : [
    2021
  ]
 }
...

So in your case, it's indeed necessary to parse date from String to manipulate it.

Best Regards,
Octavio Ranieri

[ffknob (Flávio Knob)]

What a great answer! Thank you @oranieri!