Parse Date with RFC_1123_DATE_TIME format

Elastic Stack Kibana
1

Hi All,

I'm ingesting documents to my ES cluster with a field called CREATION_TIME with format Wed, 13 Oct 2021 13:04:54 GMT.
I've tried to parse it using below mappings, but Kibana is still ignoring the value.

{
"properties": {
"CREATION_TIME": {
"format": "EEE, dd MMM yyyy hh:mm:ss zzz||EEE, dd MMM yyyy hh:mm:ss 'GMT'",
"index": true,
"ignore_malformed": true,
"store": false,
"type": "date",
"doc_values": true
}
}
}

Please your help to get the correct mappings for this date field.

Thank you!
Jesus

2

What is the error you are getting? Please share the error.

Also, it is HH, not hh for the hour.

3

Hi @leandrojmp,

I'm not getting any error, the problem is that the field is being ignored for some of my documents, even if they have the format I've specified in the mappings.

See below example:

1 - For below document CREATION_TIME field is being ignored.

>  {
>         "_index" : "tta-test",
>         "_type" : "_doc",
>         "_id" : "r0waWoYBVHApIjHukmqM",
>         "_score" : 0.0,
>         "_ignored" : [
>           "CREATION_TIME"
>         ],
>         "_source" : {
>           "ID" : 1738,
>           "ADDED_TO_TEAM_TIME" : "Wed, 20 Apr 2022 14:05:21 GMT",
>           "REVOKED_BY_NAME" : null,
>           "REVOKED_FROM_TEAM_TIME" : null,
>           "function_within_team_label" : "Team Member",
>           "CREATION_TIME" : "Wed, 20 Apr 2022 14:05:23 GMT",
>           "MODIFIED_BY_PRID" : null,
>           "MODIFICATION_TIME" : null,
>           "DELETED_BY_PRID" : null,
>           "DELETION_TIME" : null,
>           "IS_DELETED" : false,
>         }
>       },
  1. Below document is being processed without any issue.
{
        "_index" : "tta-test",
        "_type" : "_doc",
        "_id" : "n0waWoYBVHApIjHukmqM",
        "_score" : 1.0,
        "_source" : {
          "ID" : 320,
          "ADDED_TO_TEAM_TIME" : "Wed, 25 Aug 2021 11:51:00 GMT",
          "REVOKED_BY_PRID" : null,
          "REVOKED_BY_NAME" : null,
          "REVOKED_FROM_TEAM_TIME" : null,
          "function_within_team_label" : "Team Lead",
          "CREATION_TIME" : "Wed, 25 Aug 2021 11:52:25 GMT",
          "MODIFIED_BY_PRID" : null,
          "MODIFICATION_TIME" : null,
          "DELETED_BY_PRID" : null,
          "DELETION_TIME" : null,
          "IS_DELETED" : false,
        }
      }
4

Did you recreate the index with the mapping using HH ?

If I'm not wrong hh will get only 01-12 and HH will get 00-23, this seems to be the issue.

Your rejected document has the hour as 14, which won't match if you are using hh, and the one that worked has the hour as 11, which would match hh.

The time mapping needs to be HH:mm:ss, not hh:mm:ss

5

Thank you @leandrojmp, after changing the hh for HH solved my issue.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.