Ingest date with 12 hour format

Elastic Stack Elasticsearch
1

Trying to extract the date by date processor it's not successful. Still taking the normal timestamp

POST _ingest/pipeline/_simulate
{
  "pipeline":  {
	"processors": [
	  {
	    "grok": {
		   "ignore_missing": true,
		   "field": "message",
		   "patterns": ["""%{GREEDYDATA:date}"""]
			  }
		},	
      {
        "remove" : {
          "field" : "message"
        }
      },
      {
        "date" : {
          "field" : "date",
          "target_field" : "testdate",
          "formats" : ["M/d/yyyy HH:m:s a"],
          "timezone" : "Asia/Kolkata",
          "ignore_failure" : true
        }
      }
		   ]
		   },
   "docs": [
     {
       "_source": {
         "message": """7/9/2019 4:29:55 PM"""
         }
     }
   ]
  }

Result

{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_type",
        "_id" : "_id",
        "_source" : {
          "date" : "7/9/2019 4:29:55 PM",
          "testdate" : "2019-07-09T04:29:55.000+05:30"
        },
        "_ingest" : {
          "timestamp" : "2019-07-10T10:11:16.188Z"
        }
      }
    }
  ]
}
2

This looks correct to:

"testdate" : "2019-07-09T04:29:55.000+05:30"

is indeed a date coming from:

"date" : "7/9/2019 4:29:55 PM"

parsed with "M/d/yyyy HH:m:s a".

What else do you expect?

timestamp is just an information (an ingest meta data) which will not be part of your document at the end. It's the date when you ran the ingest pipeline actually.

3

Date field is Evening 4 PM but testdate field is still 04:29:55

4

Here is an example:

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "date": {
          "field": "date",
          "formats": [
            "M/d/yyyy h:m:s a"
          ]
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "date": "7/9/2019 4:29:55 PM"
      }
    }
  ]
}

Don't use "ignore_failure": true as this is hiding important error messages.
Note the change: HH is meant for 24 hours. With AM/PM you need to use 12 hours: h.

1 Like
5

Thanks .

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.