# Insert to elasticsearch from logstash IF NOT EXISTS **URL:** \ **Category:** Logstash\ **Created:** [June 3, 2022, 10:20pm UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368 "2022-06-03T22:20:27Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![seanziee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanziee/32/45151_2.png) [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Post date:** [June 3, 2022, 10:20pm UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/1 "2022-06-03T22:20:27Z")
Hi, I currently have an index that I set unique ids and sometimes there is replica data. In my use case, older data is often **more accurate** than newer data. So I would like to have logstash only insert if the \_id currently **does not** exist in the index. If it exists, the event shouldn't be sent to Elasticsearch and if it doesn't exist it should be sent. Any thoughts on how I can do this? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 3, 2022, 11:15pm UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/2 "2022-06-03T23:15:45Z")
Set the action option on the Elasticsearch output to "create" -- `create` : indexes a document, fails if a document by that id already exists in the index. Hopefully logstash does not endlessly retry 🙂 ---
**Author:** ![seanziee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanziee/32/45151_2.png) [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Post date:** [June 6, 2022, 9:28pm UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/3 "2022-06-06T21:28:12Z")
Great, I'll give that a shot! ---
**Author:** ![seanziee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanziee/32/45151_2.png) [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Post date:** [June 8, 2022, 12:22am UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/4 "2022-06-08T00:22:46Z")
looks like it worked, but yes logtash retries tirelessly. No problem because my data isn't streaming, just one time uploads thanks. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 8, 2022, 1:25am UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/5 "2022-06-08T01:25:11Z")
If it does indeed retry an event that is rejected because it already exists then that feels very much like a bug. @warkolm, do you agree? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 8, 2022, 3:48am UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/6 "2022-06-08T03:48:34Z")
Sounds like it, yeah. ---
**Author:** ![seanziee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanziee/32/45151_2.png) [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Post date:** [June 8, 2022, 5:38am UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/7 "2022-06-08T05:38:32Z")
What happens specifically is that when I'm running it as a command (not as a service), it will run through it, then terminate the pipeline and start the pipeline all over again continually. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 8, 2022, 6:09pm UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/8 "2022-06-08T18:09:58Z")
It does not retry the 409, there is just a single instance of [2022-06-08T14:08:45,101][WARN][logstash.outputs.Elasticsearch][main][ded7c2896208304e6887fb9969cdd51534a5ace0eafa08289f565caac9250dbd] Failed action {:status=\>409, :action=\>["create", {:\_id=\>"14", :\_index=\>"ecs-logstash-2022.06.08", :routing=\>nil}, {"event"=\>{"sequence"=\>0, "original"=\>"{"id": 14}"}, "@version"=\>"1", "id"=\>14, "@timestamp"=\>2022-06-08T18:08:44.971913Z, "host"=\>{"name"=\>"ip-172-31-22-149.us-east-2.compute.internal"}}], :response=\>{"create"=\>{"\_index"=\>"ecs-logstash-2022.06.08", "\_id"=\>"14", "status"=\>409, "error"=\>{"type"=\>"version\_conflict\_engine\_exception", "reason"=\>"[14]: version conflict, document already exists (current version [1])", "index\_uuid"=\>"MiVA-Ee\_QfOJhn\_C3ckXlA", "shard"=\>"0", "index"=\>"ecs-logstash-2022.06.08"}}}} ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2022, 6:10pm UTC](https://discuss.elastic.co/t/insert-to-elasticsearch-from-logstash-if-not-exists/306368/9 "2022-07-06T18:10:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.