# Error while updating nested fields using Logstash mutate

**URL:** <https://discuss.elastic.co/t/error-while-updating-nested-fields-using-logstash-mutate/271880>\
**Category:** Logstash\
**Created:** [May 1, 2021, 5:23pm UTC](https://discuss.elastic.co/t/error-while-updating-nested-fields-using-logstash-mutate/271880 "2021-05-01T17:23:53Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sunilmpatil](https://avatars.discourse-cdn.com/v4/letter/s/9dc877/32.png) [@sunilmpatil](https://discuss.elastic.co/u/sunilmpatil)\
**Post date:** [May 1, 2021, 5:23pm UTC](https://discuss.elastic.co/t/error-while-updating-nested-fields-using-logstash-mutate/271880/1 "2021-05-01T17:23:54Z")

</div>

Hi All,

I am trying to load from a SQL to Elastic using Logstash pipeline. I am trying to add new nested document using mutate, but it is overwriting existing one.

Here is my existing JSON document:

```auto
 "empid" : 12345,
 "joined_date" : "2018-12-19",
"created_dts" : "2001-03-29T07:01:01.000Z",
"projectInfo" : [
            {
            "projId":123,
            "projName":"ABC",
            "projStart":"2018-12-19"
           }
]

```

Here is the mutate block I have written:

```auto
mutate {
    add_field => {"[projectInfo][projId]" => "%{projId}"}
 add_field => {"[projectInfo][projName]" => "%{projName}"}
 add_field => {"[projectInfo][projStart]" => "%{projStart}"}
                } 

```

This mutate block should add a new project to the employee. Instead it is replacing the existing one. Please suggest

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 1, 2021, 7:12pm UTC](https://discuss.elastic.co/t/error-while-updating-nested-fields-using-logstash-mutate/271880/2 "2021-05-01T19:12:50Z")

</div>

If you are saying you have an event that has [projId], [projName], and [projStart] fields and you want to have a [projectInfo] array like

```
"projectInfo" : [
            {
            "projId":123,
            "projName":"ABC",
            "projStart":"2018-12-19"
           }
]

```

then it should be

```
mutate {
    add_field => {
        "[projectInfo][0][projId]" => "%{projId}"
        "[projectInfo][0][projName]" => "%{projName}"
        "[projectInfo][0][projStart]" => "%{projStart}"
    }
} 

```

If you want to append an entry to the array in logstash then I think you need to use a ruby filter. It is not clear what you are trying to do.

---

<div class="post-metadata">

**Author:** ![sunilmpatil](https://avatars.discourse-cdn.com/v4/letter/s/9dc877/32.png) [@sunilmpatil](https://discuss.elastic.co/u/sunilmpatil)\
**Post date:** [May 2, 2021, 3:28pm UTC](https://discuss.elastic.co/t/error-while-updating-nested-fields-using-logstash-mutate/271880/3 "2021-05-02T15:28:57Z")

</div>

Thanks @Badger for the response!

All I want to do is to not update existing nested document, but append to the existing nested documents. But the data is getting appended.

Employee was allocated to project 123, and later point I want to allocate him/her to project 456 too, without disturbing allocation to 123..

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 2, 2021, 3:54pm UTC](https://discuss.elastic.co/t/error-while-updating-nested-fields-using-logstash-mutate/271880/4 "2021-05-02T15:54:40Z")

</div>

If you want to append an entry to an array in elasticsearch when a different set of projectInfo entries are processed by logstash that is probably an elasticsearch question. It may be possible to do it using a scripted upsert (I do not know), otherwise you would have to fetch the document from elasticsearch, possibly using an elasticsearch filter, merge the additional information (probably requiring a ruby filter) and then send it back to elasticsearch.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 30, 2021, 3:55pm UTC](https://discuss.elastic.co/t/error-while-updating-nested-fields-using-logstash-mutate/271880/5 "2021-05-30T15:55:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.