# How to index a new document using existing documents?

**URL:** <https://discuss.elastic.co/t/how-to-index-a-new-document-using-existing-documents/113594>\
**Category:** Elasticsearch\
**Created:** [December 29, 2017, 1:01pm UTC](https://discuss.elastic.co/t/how-to-index-a-new-document-using-existing-documents/113594 "2017-12-29T13:01:56Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![michaelcheung](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaelcheung/32/53285_2.png) [@michaelcheung](https://discuss.elastic.co/u/michaelcheung)\
**Post date:** [December 29, 2017, 1:01pm UTC](https://discuss.elastic.co/t/how-to-index-a-new-document-using-existing-documents/113594/1 "2017-12-29T13:01:56Z")

</div>

Hello,

I have a need to index a document, with a field that computes by adding a number to an old document's field value.

For example, I had a document (id: zoneA\_userA).

```auto
{
  "_index": "online",
  "_type": "duration",
  "_id": "zoneA_userA",
...
  "_source": {
    "zone": "zoneA",
    "aggTimeSec": "1355",
...
  }
}

```

And I continuously receive numbers of logs saying additional online time of userA in zoneA. (Format: `<user> <zone> <sec>`)  
`userA zoneA 29`

I want to add value `<sec>` to the document's `aggTimeSec` value and update the document.

Now I use logstash's elasticsearch filter to query the document, and output to same document id. This works but I wonder if this is best practice? Can I do it without querying ES from logstash? I persume the flow of logstash querying elasticsearch before output is too much time load and largely degrade logstash's performance.

Regards,  
Michael

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 31, 2017, 9:57am UTC](https://discuss.elastic.co/t/how-to-index-a-new-document-using-existing-documents/113594/2 "2017-12-31T09:57:02Z")

</div>

You should be able to do this by configuring the Elasticsearch output plugin to perform a [scripted update/upsert](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-script), but I have not found any good examples showing how it is done.

---

<div class="post-metadata">

**Author:** ![michaelcheung](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaelcheung/32/53285_2.png) [@michaelcheung](https://discuss.elastic.co/u/michaelcheung)\
**Post date:** [January 2, 2018, 11:56am UTC](https://discuss.elastic.co/t/how-to-index-a-new-document-using-existing-documents/113594/3 "2018-01-02T11:56:50Z")

</div>

Thanks @Christian_Dahlqvist  
I refactored my logstash conf using following ES output.

```auto
     elasticsearch {
         hosts => ["..."]
         index => "online"
         document_type => "duration"
         document_id => "%{zone}_%{user}"
         manage_template => false
         action => "update"
         doc_as_upsert => true
         script_lang => "painless"
         script_type => "inline"
         script => '
             if (ctx._source.updateTimestamp != params.event.get("updateTimestamp")) {
                 ctx._source.aggTimeSec = Integer.parseInt(ctx._source.aggTimeSec) + Integer.parseInt(params.event.get("aggTimeSec"));
             }
         '
     }

```

Thus I can remove elasticsearch query in the filter division.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 30, 2018, 11:57am UTC](https://discuss.elastic.co/t/how-to-index-a-new-document-using-existing-documents/113594/4 "2018-01-30T11:57:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.