# Add creation (not update) time of doc using ingest

**URL:** <https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515>\
**Category:** Elasticsearch\
**Created:** [February 2, 2020, 8:17am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515 "2020-02-02T08:17:52Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\
**Post date:** [February 2, 2020, 8:17am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/1 "2020-02-02T08:17:52Z")

</div>

I'm using the following pipeline to add creation timestamp to docs but it changes the timestamp every time that I put a newer version of a same doc to elastic.

```auto
PUT _ingest/pipeline/set_creation_date
{
    "description": "Set creation date",
    "processors": [
      {
        "script": {
          "source": "ctx.created_at = new Date();"
        }
      }
    ]
}

```

Is there a way to add creation timestamp only for the first time that we add doc, and ignore the subsequent updates?

---

<div class="post-metadata">

**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\
**Post date:** [February 3, 2020, 7:35am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/2 "2020-02-03T07:35:20Z")

</div>

I resolve the issue by adding `op_type` as a query parameter when indexing docs using `PUT` or `POST` (see [docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html)). This way the elastic returns error code 409 and I catch this error when indexing documents and just pass it!

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [February 3, 2020, 3:18pm UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/3 "2020-02-03T15:18:00Z")

</div>

I am not a hundred percent sure that this solves your issues, as the whole document does not get updated in that case.

How about checking for the existence of `ctx.created_at` in the script like

```auto
if (ctx.created_at != null) {
  ctx.created_at = ...
}

```

wouldnt that work as well?

---

<div class="post-metadata">

**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\
**Post date:** [February 5, 2020, 8:40am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/4 "2020-02-05T08:40:03Z")

</div>

Thanks @spinscale. Yes the document dos not update and in my specific application it is a desirable behavior.

Very good idea, It should works, but I it tried it and I wonder why it is not to working!  
I used the following pipeline as you mentioned:

```auto
PUT _ingest/pipeline/set_creation_date
{
    "description": "Set creation date",
    "processors": [
      {
        "script": {
          "source": "if (ctx.created_at != null) {ctx.created_at = new Date();}"
        }
      }
    ]
}

```

It doesn't add the `create_at` even in the first POST request!

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [February 5, 2020, 9:22am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/5 "2020-02-05T09:22:22Z")

</div>

I think this should be `if (ctx.created_at == null)`

```auto
POST _ingest/pipeline/_simulate
{
  "docs": [
    {
      "_source": {
        "foo": "bar"
      }
    }
  ],
  "pipeline": {
    "processors": [
      {
        "script": {
          "source": "if (ctx.created_at == null) {ctx.created_at = new Date();}"
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\
**Post date:** [February 8, 2020, 6:59am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/6 "2020-02-08T06:59:39Z")

</div>

@spinscale  
That does not work neither!  
Actually I tried both in Kibana dev tools, and they didn't work!

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [February 12, 2020, 3:44pm UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/7 "2020-02-12T15:44:19Z")

</div>

please provide a fully reproducible simulate ingest pipeline call including the response. Thanks!

---

<div class="post-metadata">

**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\
**Post date:** [February 17, 2020, 6:37am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/8 "2020-02-17T06:37:08Z")

</div>

@spinscale  
Sorry for late response.

I used the following command in Kibana dev tools environment to create an ingest to set creation date:

```auto
PUT _ingest/pipeline/set_creation_date
{
    "description": "Set creation date",
    "processors": [
      {
        "script": {
          "source": "if (ctx.created_at == null) {ctx.created_at = new Date();}"
        }
      }
    ]
}

```

Then I post a doc to a text index with id 1 as follows:

```auto
POST /test/_doc/1?pipeline=set_creation_date
{"name": "a"}

```

Then I get it to see the creation time:

```auto
GET /test/_doc/1

```

It shows the doc which has a creation time like:

```auto
{
  "_index" : "test",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "a",
    "created_at" : "2020-02-17T06:36:17.423Z"
  }
}

```

but when I repeat the two step above and and the same doc with id 1, the creation time changes!

```auto
{
  "_index" : "test",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 2,
  "_seq_no" : 1,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "a",
    "created_at" : "2020-02-17T06:36:50.327Z"
  }
}

```

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [February 17, 2020, 10:35am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/9 "2020-02-17T10:35:33Z")

</div>

Hey,

if you are using the index API again, then this will be treated like a new index operation. If you want to update a document, take a look at the update API.

See [https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html)

--Alex

---

<div class="post-metadata">

**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\
**Post date:** [February 17, 2020, 10:52am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/10 "2020-02-17T10:52:45Z")

</div>

Hi, thanks @spinscale.  
I think there is a missunderstanding.  
I know the difference of POST and PUT.  
Even if I use:

```auto
PUT /test/_doc/1?pipeline=set_creation_date
{"name": "a"}

```

the end result is the same, the creation time updates every time that I update the doc.

My specific use case is that I am clawing news from some RSS feeds, hash the news body (main text) and use this hash to index news articles in elastic. The problem is that crawler runs in a cronjob every 1 hour and there may be duplicate articles with different ids in the elastic, but using the method for id generation (hash the text of article) there is (approximately) no duplicate article in the elastic but creation time changes each time that I PUT a previously added article to elastic.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [February 17, 2020, 10:53am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/11 "2020-02-17T10:53:32Z")

</div>

I did not talk about the difference between POST and PUT, but about a completely different endpoint.

---

<div class="post-metadata">

**Author:** ![alza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alza/32/50143_2.png) [@alza](https://discuss.elastic.co/u/alza)\
**Post date:** [February 17, 2020, 11:53am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/12 "2020-02-17T11:53:54Z")

</div>

Oh, yes! that is another trick, I will try it, thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 16, 2020, 11:54am UTC](https://discuss.elastic.co/t/add-creation-not-update-time-of-doc-using-ingest/217515/13 "2020-03-16T11:54:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.