# Update previosuly indexed data **URL:** \ **Category:** Logstash\ **Created:** [August 29, 2021, 8:42pm UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765 "2021-08-29T20:42:36Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Post date:** [August 29, 2021, 8:42pm UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/1 "2021-08-29T20:42:36Z")
Hello ! Is there any way to detect with logstash jdbc if some previously indexed field value has been modified and if so, replace the old value with the new modified one in the next scheduled execution? Thanks in advance! ---
**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\ **Post date:** [August 30, 2021, 5:53am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/2 "2021-08-30T05:53:34Z")
Hello Rodrigo, The best way would be to have a modification date in your table and select all entries in the jdbc input with the modification date greater than the last execution date. depending on your database you can fill the modification date in a trigger before update so you would not need to change your application. The elasticsearch output has a setting called `action` which is by default configured to `index` which means: > Indexes the specified document. If the document exists, replaces the document and increments the version. Best regards Wolfram ---
**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Post date:** [August 30, 2021, 8:39am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/3 "2021-08-30T08:39:52Z")
First of all, thank you for answering my question. I have the field modification date. so, if I put `modification_date > sql_last_value AND date > sql_last_value ` in the jdbc query and in the output `action => index` Will this replace the old values ​​with the new ones or will it create a new entry? In case he substitutes them, could you tell me how elk does that match to avoid the duplicate entry? Thank you in advance! ---
**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\ **Post date:** [August 30, 2021, 8:45am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/4 "2021-08-30T08:45:40Z")
> [@rodri.gz](#): > > Will this replace the old values ​​with the new ones or will it create a new entry? Sorry, I forgot that part ☹ Elasticsearch matches documents by ID. documents with the same ID will overwrite the existing entries. If you do not configure a custom ID it will be autogenerated so it will not overwrite. If your table contains an ID field you can use the elasticsearch output configuration for `document_id` directly: [Elasticsearch output plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-document_id) If you do not have an ID field it is harder but you could use the [fingerprint processor](https://www.elastic.co/guide/en/logstash/current/plugins-filters-fingerprint.html) on fields that never change and use this fingerprint as ID in the output plugin. ---
**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Post date:** [August 30, 2021, 9:09am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/5 "2021-08-30T09:09:26Z")
Okey. A long time ago I tried to create a unique id with the fingerprint plugin for the document\_id because it had not an id field and it works perfectly. All documents were indexed with an unique id field, but I don't understand how elk matches the document\_id the date and the value and if the value was modified change it. In my project we want to monitor the backups made in the last days but sometimes after a few days we verify if it has been done correctly if not, the value of bakups\_ok is changed manually in the database for the new number of backups and I do not know how to make logstash update it automatically. thank you and sorry for the inconvenience! ---
**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\ **Post date:** [August 30, 2021, 9:15am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/6 "2021-08-30T09:15:25Z")
Can you post an example document with sample data? ---
**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Post date:** [August 30, 2021, 9:32am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/7 "2021-08-30T09:32:44Z")
i dont know exactly what you want. we have 4 fields : backup\_ok backup\_req date and modification\_date and we are storing the sql\_last\_value with the last date when the pipeline was executed and sending the data to elk with a single output with host and index name. but i dont know how to automatically update the value if a pre-indexed value has been changed. ---
**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\ **Post date:** [August 30, 2021, 10:37am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/8 "2021-08-30T10:37:20Z")
So, the `date` is the primary key here, right? So you could use the fingerprint processor like this: ```auto fingerprint { source => ["date"] target => "id" } ``` This way, it will create an id field which you can use in the elasticsearch output from the `date` field. When you update your data in the database the date field will not be updated - only the modification\_date so the fingerprint will generate the same id. Therefore, elasticsearch will detect that a document with this id already exists and will replace it instead of creating a new document. ---
**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Post date:** [August 31, 2021, 9:46am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/9 "2021-08-31T09:46:47Z")
Thank you !! i will try it ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 28, 2021, 9:47am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765/10 "2021-09-28T09:47:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.