# How to update index in Elasticsearch
**URL:** https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773
**Category:** Logstash
**Created:** [June 13, 2021, 8:01pm UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773 "2021-06-13T20:01:55Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)
#### Post date: [June 13, 2021, 8:01pm UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773/1 "2021-06-13T20:01:55Z")
Hi All,
I have an ELK 7.6.2 stack running in the environment.
In my setup logstash is set to ingest file as an input and performs certain operations on it and creates an index in ES.
First few lines of the logstash conf file looks like below:
```auto
input {
file {
path => "/opt/gtal/ital/elasticsearch/app/logstash/stage/MQA_UAT_STATS-*.txt"
```
An index named `demo-csv-2021.06.13` (based on date) gets created as a result of the ingestion.
The file `MQA_UAT_STATS-*.txt` gets updated every hour.
I need the index `demo-csv-2021.06.13` to get updated with the new values without having to restart logstash or deleting the index.
Please help on how can I get this done? I am fairly new to ELK.
Thanks.
---
### Author:  [@Iker](https://discuss.elastic.co/u/Iker)
#### Post date: [June 14, 2021, 3:50am UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773/2 "2021-06-14T03:50:23Z")
Probably your best option is set the id of each document with a fingerprint ([Fingerprint filter plugin | Logstash Reference [7.13] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-fingerprint.html)) or another unique id, so, each time that logstash tries to index a new document, it will update the document with the same id; your pipeline will look something like this:
```auto
elasticsearch {
hosts => ["localhost:9200"]
index => "demo-csv-2021.06.13"
action => "update"
document_id => "%{my_fingerprint}"
}
```
---
### Author:  [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)
#### Post date: [June 14, 2021, 2:33pm UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773/3 "2021-06-14T14:33:41Z")
Thanks lker.
My call to ES from logstash looks like below:
```auto
elasticsearch {
hosts => ["xx-xxxx-xxxx:3045"]
user => "xxxxx"
password => "xxxxxxxx"
index => "demo-csv-%{+YYYY.MM.dd}"
```
Would fingerprint work with the daily changing index name?
Thanks
---
### Author:  [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)
#### Post date: [June 14, 2021, 7:06pm UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773/4 "2021-06-14T19:06:19Z")
Also I tried various options to set the fingerprint in my logstash conf file but it did not work out.
Can you please guide me on what additions would be required in the logstash config file with the following ES output:
```auto
elasticsearch {
hosts => ["xx-xxx-xxx:3045"]
user => "xxxxx"
password => "xxxxxxxxxxxxxxxx"
index => "demo-csv-%{+YYYY.MM.dd}"
action => "update"
document_id => "%{my_fingerprint}"
```
I kept getting the following error:
```auto
[2021-06-14T14:53:01,610][WARN][logstash.outputs.elasticsearch][main] Could not index event to Elasticsearch. {:status=>404, :action=>["update", {:_id=>"%{fingerprint}", :_index=>"demo-csv-2021.06.14", :routing=>nil, :_type=>"_doc", :retry_on_conflict=>1}, #], :response=>{"update"=>{"_index"=>"demo-csv-2021.06.14", "_type"=>"_doc", "_id"=>"%{fingerprint}", "status"=>404, "error"=>{"type"=>"document_missing_exception", "reason"=>"[_doc][%{fingerprint}]: document missing", "index_uuid"=>"5yJpaTRpSvC6SI9mTwpJwg", "shard"=>"0", "index"=>"demo-csv-2021.06.14"}}}}
```
How do I generate the fingerprint? I am not very clear on `https://www.elastic.co/guide/en/logstash/current/plugins-filters-fingerprint.html`
Thanks
---
### Author:  [@Iker](https://discuss.elastic.co/u/Iker)
#### Post date: [June 14, 2021, 10:01pm UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773/5 "2021-06-14T22:01:30Z")
If you don't have another pipeline to create the base documents this error happens because the document with the fingerprint specified doesn't exists yet, you have to add the option "doc\_as\_upsert" in your output to create or update:
```auto
elasticsearch {
hosts => ["xx-xxx-xxx:3045"]
user => "xxxxx"
password => "xxxxxxxxxxxxxxxx"
index => "demo-csv-%{+YYYY.MM.dd}"
doc_as_upsert => true
action => "update"
document_id => "%{my_fingerprint}"
```
About the fingerprint, is very easy to use, you specify which field contains a invariant text across all the document updates and just use the resulting hash as document\_id:
```auto
filter {
fingerprint {
method => "SHA256"
source => "[myfield]"
target => "[my_fingerprint]"
}
}
```
Check out this links:
[plugins-outputs-elasticsearch-action](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-action)
[plugins-outputs-elasticsearch-doc\_as\_upsert](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-doc_as_upsert)
---
### Author:  [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)
#### Post date: [June 14, 2021, 11:10pm UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773/6 "2021-06-14T23:10:40Z")
Thanks lker! It worked just fine. However I noticed that logstash consumed first couple of files and stopped consuming after 3 to 4.
Logstash is set up to start as:
`./logstash -r -f $LOGSTASH_CONFIG/mqa.conf`
---
### Author:  [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)
#### Post date: [June 16, 2021, 7:14pm UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773/7 "2021-06-16T19:14:37Z")
Thanks lker. It really helped!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 14, 2021, 7:15pm UTC](https://discuss.elastic.co/t/how-to-update-index-in-elasticsearch/275773/8 "2021-07-14T19:15:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.