# Update Dataset with Information comming delayed

**URL:** <https://discuss.elastic.co/t/update-dataset-with-information-comming-delayed/261397>\
**Category:** Elasticsearch\
**Created:** [January 18, 2021, 9:22am UTC](https://discuss.elastic.co/t/update-dataset-with-information-comming-delayed/261397 "2021-01-18T09:22:06Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![michaelpietzsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaelpietzsch/32/72828_2.png) [@michaelpietzsch](https://discuss.elastic.co/u/michaelpietzsch)\
**Post date:** [January 18, 2021, 9:22am UTC](https://discuss.elastic.co/t/update-dataset-with-information-comming-delayed/261397/1 "2021-01-18T09:22:06Z")

</div>

Hello dear elastic users,

im currently having an issue implementing a logic which updates fields after they have been stored in the elastic.

Let me try to explain this to the best of my ability:

Theres a log entrance comming in to elastic which contains fields like

- Theres a CONNECT Logline comming in. The system provides a connectionID
- A couple of seconds or minutes later a BIND Logline comes in with the same connectionID in addition now we have filled user field.  
My goal is to add this user field to the connect log line...  
.... this is the basic task...

Now heres where things really get complicated.... it is possible that multiple same connection ids occour. The data enhanchement must only work for the last operation...

any ideas on how to do this?  
best regards michael

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [January 18, 2021, 9:28am UTC](https://discuss.elastic.co/t/update-dataset-with-information-comming-delayed/261397/2 "2021-01-18T09:28:15Z")

</div>

How are you sending logs to elasticsearch ? with Logstash ?

---

<div class="post-metadata">

**Author:** ![michaelpietzsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaelpietzsch/32/72828_2.png) [@michaelpietzsch](https://discuss.elastic.co/u/michaelpietzsch)\
**Post date:** [January 18, 2021, 9:51am UTC](https://discuss.elastic.co/t/update-dataset-with-information-comming-delayed/261397/3 "2021-01-18T09:51:24Z")

</div>

They are comming from a TD agent logshipper

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [January 18, 2021, 9:55am UTC](https://discuss.elastic.co/t/update-dataset-with-information-comming-delayed/261397/4 "2021-01-18T09:55:52Z")

</div>

> [@michaelpietzsch](#):
>
> TD agent logshipper

No idea about it, but anyway, elasticsearch provide an upsert option to update existing document

> **[Update API | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#upserts)**

hat's something that logstash support OOB

> **[Elasticsearch output plugin | Logstash Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-doc_as_upsert)**

---

<div class="post-metadata">

**Author:** ![michaelpietzsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaelpietzsch/32/72828_2.png) [@michaelpietzsch](https://discuss.elastic.co/u/michaelpietzsch)\
**Post date:** [January 18, 2021, 10:07am UTC](https://discuss.elastic.co/t/update-dataset-with-information-comming-delayed/261397/5 "2021-01-18T10:07:21Z")

</div>

Would the enrich processor be a viable option?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 15, 2021, 10:07am UTC](https://discuss.elastic.co/t/update-dataset-with-information-comming-delayed/261397/6 "2021-02-15T10:07:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
