# Insert in to existing index using some condition
**URL:** https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848
**Category:** Logstash
**Created:** [June 4, 2020, 8:55pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848 "2020-06-04T20:55:20Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [June 4, 2020, 8:55pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/1 "2020-06-04T20:55:20Z")
Here is what I am trying to do
I have a existing index with
```
{
index= my_index
id=1
fname= xyz
lname = abc
emp_num=123
loction=usa
}
```
now i am reading new data from new source A
which have fname, lname, address
I would like to insert that address in to elk record if fname and lname match
```
{
index=my_index
id=1
fname= xyz
lname = abc
emp_num=123
loction=usa
address=new_address_here
}
```
how do I do this? would like to use logstash. and source is sqlserver. connection string is already establish.
---
### Author:  [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [June 5, 2020, 2:43pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/2 "2020-06-05T14:43:51Z")
any idea?
---
### Author:  [@ptamba](https://discuss.elastic.co/u/ptamba)
#### Post date: [June 5, 2020, 2:49pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/3 "2020-06-05T14:49:23Z")
after your jdbc , you can use elasticsearch filter to query the existing document in ES, add the new field and use update mode on elasticsearch output.
---
### Author:  [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [June 5, 2020, 6:33pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/4 "2020-06-05T18:33:38Z")
can I simple use translate filter?
where my dictionary file is something like this.
fname:lnama:address
---
### Author:  [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [June 5, 2020, 9:18pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/5 "2020-06-05T21:18:43Z")
is this syntax looks ok?
I am getting
[0] "\_elasticsearch\_lookup\_failure"
```
elasticsearch {
hosts => ["elkdev01:9200"]
index => "fw_lifespan"
user => "${elastic_user}"
password => "${elastic_password}"
#here I want to also compare fw_vendor.keyword
query => 'fw_version.keyword:"%{[firmware]}"'
# if match would like to take this two field from fw_lifespan index
fields => ["fw_recommended","fw_expiration_date"] }
}
```
---
### Author:  [@ptamba](https://discuss.elastic.co/u/ptamba)
#### Post date: [June 6, 2020, 4:24am UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/6 "2020-06-06T04:24:43Z")
> [@elasticforme](#):
>
> ```auto
> user => "${elastic_user}"
> password => "${elastic_password}"
>
> ```
the syntax seems ok to me. do you have user and password stored as local variables ? does the log showed failure reason ?
---
### Author:  [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [June 8, 2020, 1:55pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/7 "2020-06-08T13:55:14Z")
yes that username/password works. I can connect to it
---
### Author:  [@ptamba](https://discuss.elastic.co/u/ptamba)
#### Post date: [June 8, 2020, 2:06pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/8 "2020-06-08T14:06:43Z")
no errors in the log? maybe increase log level to debug
---
### Author:  [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [June 8, 2020, 2:22pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/9 "2020-06-08T14:22:19Z")
no error. it is printing this
[0] "\_elasticsearch\_lookup\_failure"
this shows up only on few document not on all documents
---
### Author:  [@ptamba](https://discuss.elastic.co/u/ptamba)
#### Post date: [June 8, 2020, 2:35pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/10 "2020-06-08T14:35:55Z")
are you able to search the failed docs in elasticsearch?
if yes, I would increase the log level in loglevel to debug or trace to see why it fails
---
### Author:  [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [June 8, 2020, 3:28pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/11 "2020-06-08T15:28:24Z")
After lot of testing, it is working now. here is final configuration
```
filter{
elasticsearch {
hosts => ["hostname:9200"]
index => "fw_lifespan"
user => "${elastic_user}"
password => "${elastic_password}"
query => "fw_version:%{firmware} AND fw_vendor:%{vendor}"
fields => { "fw_recommended" => "fw_recommended"
"fw_expiration_date" => "fw_expiration_date" }
}
}
```
two new field will come from fw\_lifespan index to new index if it match firmware and vendor.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2020, 3:28pm UTC](https://discuss.elastic.co/t/insert-in-to-existing-index-using-some-condition/235848/12 "2020-07-06T15:28:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.