# Adding scripted fields to document source

**URL:** <https://discuss.elastic.co/t/adding-scripted-fields-to-document-source/298552>\
**Category:** Elasticsearch\
**Tags:** painless\
**Created:** [March 1, 2022, 7:26pm UTC](https://discuss.elastic.co/t/adding-scripted-fields-to-document-source/298552 "2022-03-01T19:26:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![andersnelson](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@andersnelson](https://discuss.elastic.co/u/andersnelson)\
**Post date:** [March 1, 2022, 7:26pm UTC](https://discuss.elastic.co/t/adding-scripted-fields-to-document-source/298552/1 "2022-03-01T19:26:43Z")

</div>

I am trying to move away from scripted fields in our Elasticsearch cluster. I have updated our Logstash config files to add the relevant fields to new incoming documents, but am having trouble figuring out how to update existing documents. I set up an ingest pipeline that runs a Painless script to add the fields if they don't exist in a document. I am running an update\_by\_query in Kibana dev tools like this:

```auto
POST log-logname*/_update_by_query?pipeline=Add-hr-day-fields&conflicts=proceed
{
  "query":{
    "range":{
      "@timestamp": {
        "gte": "2022-02-26",
        "lte": "2022-02-28"
        
      }
    }
  }
}

```

This does add the fields to the documents, but it seems to keep running after all the relevant documents have been updated. I am checking that with this command:  
`GET _tasks?detailed=true&actions=*byquery`

I think it has something to do with version conflicts, since I see version conflict errors if I don't add `conflicts=proceed ` to the command.

So, is running an update by query the preferred way to do this, or should I do it another way? What causes the version conflicts and how can I avoid them?

Thanks much.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 1, 2022, 10:15pm UTC](https://discuss.elastic.co/t/adding-scripted-fields-to-document-source/298552/2 "2022-03-01T22:15:48Z")

</div>

Welcome to our community! 😃

> [@andersnelson](#):
>
> So, is running an update by query the preferred way to do this

It looks like the best option for you, yes.

> [@andersnelson](#):
>
> What causes the version conflicts and how can I avoid them?

We'd probably to see some of the errors you get there to comment accurately.

---

<div class="post-metadata">

**Author:** ![andersnelson](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@andersnelson](https://discuss.elastic.co/u/andersnelson)\
**Post date:** [March 2, 2022, 2:01pm UTC](https://discuss.elastic.co/t/adding-scripted-fields-to-document-source/298552/3 "2022-03-02T14:01:31Z")

</div>

Thanks for the reply! Here is a representative error:

```auto
  "failures" : [
    {
      "index" : "log-logname",
      "type" : "_doc",
      "id" : "rtgor16o001m01_Se0/0/0:1.64.1644254100",
      "cause" : {
        "type" : "version_conflict_engine_exception",
        "reason" : "[rtgor16o001m01_Se0/0/0:1.64.1644254100]: version conflict, required seqNo [5356283], primary term [3]. current document has seqNo [9338852] and primary term [5]",
        "index_uuid" : "XNeenpEfSrWEBSqhbnBohg",
        "shard" : "0",
        "index" : "log-logname"
      },
      "status" : 409
    },

```

I get many of these if I don't run the update by query with the `conflicts=proceed` option.

Also, sometimes I see an error about trying to create too many scroll contexts, should I just add the option to create more?

```auto
        "type" : "exception",
        "reason" : "Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."
      },

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 30, 2022, 2:01pm UTC](https://discuss.elastic.co/t/adding-scripted-fields-to-document-source/298552/4 "2022-03-30T14:01:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.