# Old doc versions will not be deleted

**URL:** <https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061>\
**Category:** Elasticsearch\
**Created:** [August 24, 2020, 7:13am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061 "2020-08-24T07:13:32Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Messias\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/messias_2/32/74381_2.png) [@Messias\_2](https://discuss.elastic.co/u/Messias_2)\
**Post date:** [August 24, 2020, 7:13am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/1 "2020-08-24T07:13:33Z")

</div>

Hi,

in this discuss [Deleting old versions](https://discuss.elastic.co/t/deleting-old-versions/4689/3) stands if you use the same ID for a doc: ... each time you index a doc with the same ID, the old version will be marked as deleted, and will no longer be visible.

we have now with the Kibana Version 7.9.0 the "issue" that the discover shows all versions (the actual and old). These 'deleted' doc will not be removed after more than 14 days, even if we do a \_forcemerge.

How can we delete every old version inside one index? And is it possible to disable the old version shown in Kibana? In every request in discovery the version: true is set. And I have found no option to disable it.

Best regards  
Messias

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 24, 2020, 7:50am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/2 "2020-08-24T07:50:59Z")

</div>

Can you please show an example of these duplicate documents? Make sure you include the document id and index name.

---

<div class="post-metadata">

**Author:** ![Messias\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/messias_2/32/74381_2.png) [@Messias\_2](https://discuss.elastic.co/u/Messias_2)\
**Post date:** [August 24, 2020, 8:31am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/3 "2020-08-24T08:31:13Z")

</div>

Hi,

@Christian_Dahlqvist: You understand it wrong there are no duplicates. Only different version of one document.

If you search with: /\_search?version=true you will get like this:

```auto
{
    "_index": "index-2020.08.09",
    "_type": "_doc",
    "_id": "ID#1",
    "_score": 1.0,
    "_version": 1,
    "_source": {
        "@timestamp": "2020-08-09T16:48:13",
        SOMEDATA
}

{
    "_index": "index-2020.08.09",
    "_type": "_doc",
    "_id": "ID#1",
    "_score": 1.0,
    "_version": 2,
    "_source": {
        "@timestamp": "2020-08-09T16:48:13",
        UPDATEDDATA
}

```

the older version (in this case 1) do not needed.

best regards  
Messias

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 24, 2020, 8:32am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/4 "2020-08-24T08:32:12Z")

</div>

Is that the actual document `_id` for each one?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 24, 2020, 8:34am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/5 "2020-08-24T08:34:13Z")

</div>

That is what I meant by duplicate. Are the ids identical? Do you use routing during indexing?

---

<div class="post-metadata">

**Author:** ![Messias\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/messias_2/32/74381_2.png) [@Messias\_2](https://discuss.elastic.co/u/Messias_2)\
**Post date:** [August 24, 2020, 9:05am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/6 "2020-08-24T09:05:15Z")

</div>

Hi,

@warkolm: this is only a dummy for ID

@Christian_Dahlqvist: the IDs are identical and no routing is used.

I do not understand why the old version is not be deleted. And have one a option / idea how you can disable the "version: true" in Kibana?

Best regards  
Messias

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 24, 2020, 9:14am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/7 "2020-08-24T09:14:43Z")

</div>

How many shards does the index in question have? If you are not explicitly using routing are you by any chance using parent-child?

---

<div class="post-metadata">

**Author:** ![Messias\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/messias_2/32/74381_2.png) [@Messias\_2](https://discuss.elastic.co/u/Messias_2)\
**Post date:** [August 24, 2020, 9:26am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/8 "2020-08-24T09:26:16Z")

</div>

@Christian_Dahlqvist: only 2 shards. I have never used a parent-child feature. The only thing is where this happens is I use the update API and do not a complete new index.

Thanks for your input i will test some things.

Best regards  
Messias

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 24, 2020, 10:02am UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/9 "2020-08-24T10:02:24Z")

</div>

Have never seen this problem before so am out of ideas.

---

<div class="post-metadata">

**Author:** ![dominique.bejean](https://avatars.discourse-cdn.com/v4/letter/d/dc4da7/32.png) [@dominique.bejean](https://discuss.elastic.co/u/dominique.bejean)\
**Post date:** [August 24, 2020, 12:34pm UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/10 "2020-08-24T12:34:02Z")

</div>

Hi Messias,

As it is explained in the article you are referring to, if documents are updated with the same '\_id' value, a new document is created and the old one is internally flagged by lucene as deleted and never returned in search results. Future segment merge operations will definitivelly remove if.

If updates create new documents with dedicated \_id value, \_forcemerge won't make them disappear.

How are these documents updated and what is this 14 days retention period you are speaking about ?  
The only place a find this kind of retention is in the self hosted gitlab documentation about the elasticsearch integration.  
[https://docs.gitlab.com/ee/integration/elasticsearch.html#trigger-the-reindex-via-the-elasticsearch-administration](https://docs.gitlab.com/ee/integration/elasticsearch.html#trigger-the-reindex-via-the-elasticsearch-administration)

Regards

Dominique

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 24, 2020, 9:05pm UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/11 "2020-08-24T21:05:37Z")

</div>

> [@Messias\_2](#):
>
> @warkolm: this is only a dummy for ID

Can you please show the exact IDs and as much of the original document as you can.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 21, 2020, 9:05pm UTC](https://discuss.elastic.co/t/old-doc-versions-will-not-be-deleted/246061/12 "2020-09-21T21:05:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
