# Scripted upsert is failing in Elasticsearch output

**URL:** https://discuss.elastic.co/t/scripted-upsert-is-failing-in-elasticsearch-output/349057
**Category:** Logstash
**Tags:** painless
**Created:** [December 11, 2023, 3:02pm UTC](https://discuss.elastic.co/t/scripted-upsert-is-failing-in-elasticsearch-output/349057 "2023-12-11T15:02:26Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![gshankar-elastic](https://avatars.discourse-cdn.com/v4/letter/g/f04885/32.png) [@gshankar-elastic](https://discuss.elastic.co/u/gshankar-elastic)
#### Post date: [December 11, 2023, 3:02pm UTC](https://discuss.elastic.co/t/scripted-upsert-is-failing-in-elasticsearch-output/349057/1 "2023-12-11T15:02:26Z")

</div>

I am using an indexed script in the output to transform the `event` data like this:

```auto
input {
    kafka {
        bootstrap_servers => "kafka.localhost.com:9092"
        topics => ["enriched"]
    }
}

filter {
    json {
        source => "message"
        remove_field => ["message"]
    }

    ruby {
        code => "

            id = event.get('[enrichments][id]')
            dedup_id = event.get('dedup_id')

            event.set('[@metadata][_id]', id)
            event.set('[@metadata][dedup_id]', dedup_id)

        "
    }    
}

output {
    elasticsearch {
        hosts => ["elasticsearch:9200"]
        index => "dedup_store"
        document_id => "%{[@metadata][dedup_id]}"
        action => "update"
        scripted_upsert => true
        script_lang => ""
        script_type => "indexed"
        script => "dedup_upsert_script"
    }
}

```

And here is the indexed script:

```auto
curl -X POST "http://localhost:9200/_scripts/dedup_upsert_script" -H 'Content-Type: application/json' -d '
{
  "script": {
    "lang": "painless",
    
    "source": "if (!ctx._source.containsKey(\"stories\")) { ctx._source.stories = []; } def storyId = params.event['@metadata']['_id']; def createdAt = params.event['created_at']; if (ctx._source.stories.find(s -> s[\"id\"] == storyId) == null) { def newStory = new HashMap(); newStory[\"id\"] = storyId; newStory[\"created_at\"] = createdAt; ctx._source.stories.add(newStory); }"

  }
}'

```

I am getting following error:

> :response=\>{"update"=\>{"\_index"=\>"dedup\_store", "\_type"=\>"\_doc", "\_id"=\>"B51661CB86F58DF6865EBF00CFC8BDE7\_4chan", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"failed to execute script", "caused\_by"=\>{"type"=\>"script\_exception", "reason"=\>"runtime error", "script\_stack"=\>["storyId = params.event['@metadata']['\_id']; def ", " ^---- HERE"], "script"=\>"dedup\_upsert\_script", "lang"=\>"painless", "position"=\>{"offset"=\>97, "start"=\>75, "end"=\>123}, "caused\_by"=\>{"type"=\>"null\_pointer\_exception", "reason"=\>"cannot access method/field [normalizeIndex] from a null def reference"}

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 8, 2024, 3:02pm UTC](https://discuss.elastic.co/t/scripted-upsert-is-failing-in-elasticsearch-output/349057/2 "2024-01-08T15:02:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.