Date Field with Millis stored as double leading to error

Elastic Stack Elasticsearch
1

Hey there,

I recently got this strange problem in a new index:
Basic info:
Elasticsearch Version: 7.13.2
Nodes: 1 Node (localhost in Dev-Env)
Middleware: [Elasticsearch-PHP] API-Wrapper

The index got a date field and a numeric field to count up with the explicit mapping:

//PHP-Syntax but should be readable
"click_count" => [
          "type" => "long",
],
"created" => [
          "type" => "date",
          "format" => "strict_date_optional_time||epoch_millis"
],

If a document gets indexed with a epochMillis value everything is fine.
However if want to increase the counter by an update call with scripted update:

POST test/_update/sZwXXXoBfo59-9IusoIu
{
  "script" : {
    "source": "ctx._source.click_count += params.count",
    "lang": "painless",
    "params" : {
      "count" : 1
    }
  }
}

I am getting the following Exception:

..."type":"mapper_parsing_exception","reason":"failed to parse field [created] of type [date] in document with id 'sZwXXXoBfo59-9IusoIu'. Preview of field's value: '1.625059471792E12'","caused_by":{"type":"illegal_argument_exception","reason":"failed to parse date field [1.625059471792E12] with format [strict_date_optional_time||epoch_millis]"...

If i use the following script it works:

POST test/_update/sZwXXXoBfo59-9IusoIu
{
  "script" : {
    "source": "ctx._source.click_count += params.count; if(ctx._source.created instanceof double) {ctx._source.created = (long)ctx._source.created}",
    "lang": "painless",
    "params" : {
      "count" : 1
    }
  }
}

I am wondering now is this the intended behaviour? Feels a bit hacky?
What i am doing wrong? What is best practice here?

Thank you for your tipps and time.

Best Regards
Bertram

2

Can you share a sample document and how it is stored in Elasticsearch? You may want to store dates as strings in a human readable format, instead as epoch milliseconds/seconds...

3

Hey thank you for your reply,

This is an example document (there are some more fields, but those are obmitted as they don't make any difference)

{
   "click_count": 1,
   "created": 1625061253285
}

If I get this indexed document:

{
   "_index": "myindex",
   "_type": "_doc",
    //...
    "_id": "spwyXXoBfo59-9Iu4YIO",
    "_source": {
        "click_count": 1,
        "created": 1625061253285
    }
}

If i use String based Date Values instead, everything works fine on update and no error occurs in any case:

{
   "click_count": 1,
   "created": "2021-07-01T09:25:10.129+02:00"
}

If I get this indexed document:

{
   "_index": "myindex",
   "_type": "_doc",
    //...
    "_id": "spwyXXoBfo59-9Iu4YIO",
    "_source": {
        "click_count": 1,
        "created": "2021-07-01T09:25:10.129+02:00"
    }
}

You are right, human readable date may be better solution and i may keep it anyway. However i want to understand why elasticsearch fails to reindex a document with epochMilli date field value, without the special treatment on updates.

4

So I tried the below on 7.13.1 and it worked for me

DELETE test

PUT test 
{
  "mappings": {
    "properties": {
      "created" : {
        "type": "date",
        "format": "strict_date_optional_time||epoch_millis"
      }
    }
  }
}

PUT test/_doc/1
{
   "click_count": 1,
   "created": 1625061253285
}

POST test/_update/1
{
  "script" : {
    "source": "ctx._source.click_count += params.count; if(ctx._source.created instanceof double) {ctx._source.created = (long)ctx._source.created}",
    "lang": "painless",
    "params" : {
      "count" : 1
    }
  }
}

GET test/_doc/1

is there any difference compared to your example that I am missing?

5

This is already the workarround. Which is working. But I do not understand that elastic is not capable to run the following with no error:

POST test/_update/1
{
  "script" : {
    "source": "ctx._source.click_count += params.count",
    "lang": "painless",
    "params" : {
      "count" : 1
    }
  }
}

And if the workarround is intended to be used, why does elasticsearch support the format "epoch_millis" at all, if you have to look and take care on every update of documents with casting.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.