# Update/Add multiple records in elastic using logstash

**URL:** <https://discuss.elastic.co/t/update-add-multiple-records-in-elastic-using-logstash/164316>\
**Category:** Logstash\
**Created:** [January 15, 2019, 1:44pm UTC](https://discuss.elastic.co/t/update-add-multiple-records-in-elastic-using-logstash/164316 "2019-01-15T13:44:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![amit.patra](https://avatars.discourse-cdn.com/v4/letter/a/34f0e0/32.png) [@amit.patra](https://discuss.elastic.co/u/amit.patra)\
**Post date:** [January 15, 2019, 1:44pm UTC](https://discuss.elastic.co/t/update-add-multiple-records-in-elastic-using-logstash/164316/1 "2019-01-15T13:44:37Z")

</div>

We have a requirement,

We have multiple json data like below :

{  
"\_id": "1",  
"\_index": "vnfsdb-20181214",  
"\_type": "l4",  
"\_source": {  
"@timestamp": "2018-10-01",  
"Flagvalue": "1",  
"Terminaltype": "T2",  
"ObjectFlag": "0",  
"Modelname2": "demo2",  
"Interceptionsystem": "I\_Sat",  
"CallingIMEI": "356789831620581",  
"Langfound": "bengali",  
"Lastmodifieddatetime": "2018-10-01"  
}  
}

We want two operation here from a single logstash conf file,

1. insert the json file into a index call "alert-20181214".

2. We want to store the value of only \_id in a variable and update the document present in "vnfsdb-20181214 " index.

We want to add two fields into "vnfsdb-20181214 " index based on \_id which i stored in a variable.

Can anyone please provide the conf file to make this operation and save my day..

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![amit.patra](https://avatars.discourse-cdn.com/v4/letter/a/34f0e0/32.png) [@amit.patra](https://discuss.elastic.co/u/amit.patra)\
**Post date:** [January 16, 2019, 2:41pm UTC](https://discuss.elastic.co/t/update-add-multiple-records-in-elastic-using-logstash/164316/2 "2019-01-16T14:41:06Z")

</div>

Update:

We want to update via logstash output.

output {  
elasticsearch {  
hosts =\> ["192.168.2.79:9200"]  
index =\> "test"  
document\_type =\> "type1"  
action =\> "update"  
script\_lang =\> "painless"  
#script\_lang =\> "groovy"  
script\_type =\> "inline"  
document\_id =\> "1"  
doc\_as\_upsert =\> true  
script =\> 'ctx.\_source.counter = 786'  
}  
}

We want to set counter value 786 in test index .

GET test/type1/1

{  
"\_index": "test",  
"\_type": "type1",  
"\_id": "1",  
"\_version": 9,  
"found": true,  
"\_source": {  
"counter": 555  
}  
}

Please help.

---

<div class="post-metadata">

**Author:** ![Chris\_Lyons](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_lyons/32/48107_2.png) [@Chris\_Lyons](https://discuss.elastic.co/u/Chris_Lyons)\
**Post date:** [January 16, 2019, 4:20pm UTC](https://discuss.elastic.co/t/update-add-multiple-records-in-elastic-using-logstash/164316/3 "2019-01-16T16:20:27Z")

</div>

This is an example of how to achieve your first post:

> input {  
> \<your\_input\_plugin\>{  
> .....  
> add\_field =\> {"[@metadata][index]" =\> "alert-20181214"}  
> }  
> }
> 
> #Clone the event  
> clone {id =\> "REMEDY\_TRANS\_CLONE" clones =\> ["new-type"]}}
> 
> #Clean up the event you want stripped down and sent to new index  
> if [type] == "new-type" {  
> #Use prune to remove all fields but the ones you want  
> prune { whitelist\_names =\> [] }  
> #Repalce your index value with the new destination  
> mutate { replace =\> { "[@metadata][index]" =\> "vnfsdb-20181214" } }  
> }
> 
> output{  
> {  
> .....  
> index =\> "%{[@metadata][index]}"  
> }  
> }

Where is the counter value coming from? If you control the document\_id you are sending to you can avoid scripting and just update the field with whatever value gets applied in the pipeline. Preserve your id field in the prune whitelist and apply to the document\_id =\> "%{..}" in the output with your update action. Now if you are incrementing every time that document is updated you would need to leverage scripting if the count isn't already in your document. Something like:

> ctx.\_source.counter+=1

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 13, 2019, 4:20pm UTC](https://discuss.elastic.co/t/update-add-multiple-records-in-elastic-using-logstash/164316/4 "2019-02-13T16:20:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.