# Scripted Upsert with Logstash ES Output Plugin - overwrites array data with latest event instead of update on reconnection

**URL:** <https://discuss.elastic.co/t/scripted-upsert-with-logstash-es-output-plugin-overwrites-array-data-with-latest-event-instead-of-update-on-reconnection/232547>\
**Category:** Logstash\
**Tags:** painless\
**Created:** [May 14, 2020, 4:49am UTC](https://discuss.elastic.co/t/scripted-upsert-with-logstash-es-output-plugin-overwrites-array-data-with-latest-event-instead-of-update-on-reconnection/232547 "2020-05-14T04:49:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![VIVEK\_SHARMA3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vivek_sharma3/32/68298_2.png) [@VIVEK\_SHARMA3](https://discuss.elastic.co/u/VIVEK_SHARMA3)\
**Post date:** [May 14, 2020, 4:49am UTC](https://discuss.elastic.co/t/scripted-upsert-with-logstash-es-output-plugin-overwrites-array-data-with-latest-event-instead-of-update-on-reconnection/232547/1 "2020-05-14T04:49:22Z")

</div>

**Scenario:** We are adding object to array via logstash to elasticsearch using logstash's ES output plugin. Both are on version 6.8. It is a scripted upsert using stored script on Elasticsearch and here is the stored script on ES:

```auto
"add_script" : {
        "lang" : "painless",
        "source" : "if (ctx._source.tags != null ) { ctx._source.tags.add(params.event.get('tags')[0])} else {ctx._source.tags = params.event.get('tags')} "
      }

```

Logstash output is:

```auto
       output {
            elasticsearch {
                    "action" =>"update"
                    "hosts" => " *****"
                    "index" => "%{index}"
                    "document_id" => "%{cid}"
                    "scripted_upsert" => true
                    "upsert" => ""
                    script_lang => ""
                    script_type => "indexed"
                    script => "add_script"
                    timeout => 120

           }
        } 

```

With above two systems we are trying to push tags object into an array if it exists, else create a new tag array if it doesn't. Simple stuff so far!

**Issue:** Under load testing where ES endpoint become momentarily unavailable (and this issue is for later discussions) or a logstash restart with few events in queue, the moment connection is restored, the script ignores the != null part and directly overwrites the entire array of tags. For instance, if the tags array was initially like this in ES:

```auto
    {
        cid: 1,
        tags: [{
                id: 1,
                "tag": "test_tag1"
            },
            {
                id: 2,
                "tag": "test_tag2"
            },
            {
                id: 3,
                "tag": "test_tag3"
            }
        ]
    }

```

and the last event was that came in for update was

```auto
    tags: [{
            id: 4,
            "tag": "test_tag4"
        }]

```

then instead of pushing this array the end document looks like this:

```auto
    {
        cid: 1,
        tags: [{
            id: 4,
            "tag": "test_tag4"
        }]
    }

```

This happens only when there is connection restore to ES, as mentioned earlier either when ES encounter host unreachable error or logstash restarts while some events are still in queue and they start getting applied as soon it starts.

So far few things which we have played around with **no luck** are

1. Reducing bulk size and number of workers in logstash pipeline - this was done keeping in mind that we may be overwhelming ES.
2. Increase the retry\_interval to 30s from 2s, this was done with a theory that when we reconnect to ES and the script executes, there is a window of time when ES return no records for matching documents and thus thinks that document key is empty and goes ahead and runs the else part, overwriting the entire array. So to give ES sometime to warmup, increased this.

Help needed as soon as possible to rectify this situation, as we just stopped a major product feature release due to this issue and we are not able to figure out any way to get out of this. Why is logstash on connection not able to honour the null check `if (ctx._source.tags != null )` on reconnection? Can we call this as bug in ElasticSeach?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 11, 2020, 4:49am UTC](https://discuss.elastic.co/t/scripted-upsert-with-logstash-es-output-plugin-overwrites-array-data-with-latest-event-instead-of-update-on-reconnection/232547/2 "2020-06-11T04:49:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.