# Indexing same document twice

**URL:** <https://discuss.elastic.co/t/indexing-same-document-twice/59306>\
**Category:** Elasticsearch\
**Created:** [August 30, 2016, 12:24pm UTC](https://discuss.elastic.co/t/indexing-same-document-twice/59306 "2016-08-30T12:24:18Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![egens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/egens/32/11705_2.png) [@egens](https://discuss.elastic.co/u/egens)\
**Post date:** [August 30, 2016, 12:24pm UTC](https://discuss.elastic.co/t/indexing-same-document-twice/59306/1 "2016-08-30T12:24:18Z")

</div>

Suppose I have some document

```auto
id=1
{
"foo":"bar"
}

```

What happens if I'm trying to index the same document? Will it be reindexed? Is it the same as if some field changed? (I see that version number grows so I expect so)

```auto
id=1
{
"foo":"baz"
}

```

Is there a nice way to check if I'm going to index duplicate document?

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [August 30, 2016, 12:55pm UTC](https://discuss.elastic.co/t/indexing-same-document-twice/59306/2 "2016-08-30T12:55:55Z")

</div>

1. Use ``` to mark code blocks so discuss will preserve the indentation.
2. Have a look at [version\_type](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#_version_types), particularly internal vs external. The other version types are really more corner case things. Don't use `force`. It is evil.
3. On that same page there is documentation for the `_create` action which will create documents if they haven't been seen. That'll catch a duplicate document.
4. Putting `id` in the document isn't the same as putting it on the URL. The url is authoritative. The one in the document is just another field.

---

<div class="post-metadata">

**Author:** ![egens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/egens/32/11705_2.png) [@egens](https://discuss.elastic.co/u/egens)\
**Post date:** [August 30, 2016, 1:35pm UTC](https://discuss.elastic.co/t/indexing-same-document-twice/59306/3 "2016-08-30T13:35:05Z")

</div>

Thanks for the answer @nik9000. By adding `id` to json I meant setting same internal `id`.

Create won't add document even if it has changed as I understand. And I need that.

The task is given a large flow of documents with the comparable number of unique ones, duplicates and ones with something changed:

1. Index document if it is not exist.
2. Update fields if the new one have some changes.  
_3. Do nothing if it is duplicate._

The third item I'm having trouble with. Update with doc\_as\_upsert gives me 1. and 2. but as I understand 3. isn't there – ES do reindex for duplicates (?).

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [August 30, 2016, 1:55pm UTC](https://discuss.elastic.co/t/indexing-same-document-twice/59306/4 "2016-08-30T13:55:41Z")

</div>

Are you looking for [detect\_noop](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#_detecting_noop_updates)? That is on by default since 2.0 I believe.

The updates using a partial document merge the new document into the old document. If that works for you then I think all you have to do is use a partial document and `doc_as_upsert`?

---

<div class="post-metadata">

**Author:** ![egens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/egens/32/11705_2.png) [@egens](https://discuss.elastic.co/u/egens)\
**Post date:** [August 30, 2016, 8:46pm UTC](https://discuss.elastic.co/t/indexing-same-document-twice/59306/5 "2016-08-30T20:46:22Z")

</div>

What do you mean by "use a partial document"?

So if detect\_noop works should version number grow when indexing same data? (using 2.3, will check if it is enabled but it should be)  
As I understand reindexing is costly operation. Detecting noops isn't?  
Getting document and checking by myself is worse than using detect\_noop?

upd. Just tested. I've got 2.3.3.

```auto
POST test/test/1/_update
{
  "doc" : {
      "test" : "test"
  }
}

```

This really doesn't increase version number. So it writes nothing? And bazillion duplicate updates won't kill my SSDs?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:23pm UTC](https://discuss.elastic.co/t/indexing-same-document-twice/59306/6 "2017-07-05T22:23:43Z")

</div>