Using ingestion pipeline to add metadata information to documents

[nidheeshb (Nidheesh)]

I am trying to use an Ingestion pipeline to add _timestamp field to the documents.

While using the ingestion simulate API , the response contains the _timestamp field that gets added due to the pipeline processor

POST _ingest/pipeline/_simulate?verbose
{
  "pipeline": {
    "description": "Adds a timestamp field at the current time",
    "processors": [
      {
        "set": {
          "field": "_timestamp",

          "value": "{{_ingest.timestamp}}"
        }
      }
    ]
  },
  "docs":[
    {
      "_index":"newindex",
      "_type": "document",
      "_id":"1",
      "_source":{
        "example":"data3"
      }
    }
    ]
}

But after issuing a write request using the pipeline, the _timestamp field fails to get added to the document source. Changing the field name from _timestamp to timestamp seems to work though.

[msimos (Mike Simos)]

_timestamp was removed in 5.x:

https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_50_mapping_changes.html#_literal__timestamp_literal_and_literal__ttl_literal

Also it appears that _timestamp is still a reserved name. So this is why it works when you just use timestamp. You can see the error by adding a document with a field name called _timestamp.