# Doc's are getting deleted automatically

**URL:** <https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879>\
**Category:** Elasticsearch\
**Created:** [July 6, 2018, 9:30am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879 "2018-07-06T09:30:53Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![kish](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@kish](https://discuss.elastic.co/u/kish)\
**Post date:** [July 6, 2018, 9:30am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/1 "2018-07-06T09:30:53Z")

</div>

Doc in the index are getting deleted automatically from one particular index alone i.e immediately . I am able to see in indices output and very next second it is getting deleted.

One the ES logs i am to see as below when deletion happened.

> [2018-07-06T14:51:47,067][DEBUG][o.e.i.r.TransportDeleteByQueryAction] [cscfhcalloam002] executing initial scroll against [register][register]  
> [2018-07-06T14:51:47,072][DEBUG][o.e.i.r.TransportDeleteByQueryAction] [cscfhcalloam002] scroll returned [1] documents with a scroll id of [DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAA4RFmFkSHo1VVhjVG1HZjZBYkp6ZEF4Z3cAAAAAAAAOEBZhZEh6NVVYY1RtR2Y2QWJKemRBeGd3AAAAAAAAEuYWeEJ5aXV3NWtRcW1TWlc5RGpiQUFwdwAAAAAAABLoFnhCeWl1dzVrUXFtU1pXOURqYkFBcHcAAAAAAAAS5xZ4QnlpdXc1a1FxbVNaVzlEamJBQXB3]  
> [2018-07-06T14:51:47,073][DEBUG][o.e.i.r.TransportDeleteByQueryAction] [cscfhcalloam002] sending [1] entry, [50b] bulk request  
> [2018-07-06T14:51:47,293][DEBUG][o.e.i.r.TransportDeleteByQueryAction] [cscfhcalloam002] Freed [5] contexts

Problematic index ,  
root@cscfhcalloam002\> curl -XGET 'localhost:9200/noderegister/\_mapping?pretty'

> {  
> "noderegister" : {  
> "mappings" : {  
> "register" : {  
> "properties" : {  
> "Address" : {  
> "type" : "ip"  
> },  
> "NodeName" : {  
> "type" : "keyword"  
> },  
> "NodeType" : {  
> "type" : "keyword"  
> },  
> "Uuid" : {  
> "type" : "keyword"  
> }  
> }  
> },  
> "noderegister1" : {  
> "properties" : {  
> "Address" : {  
> "type" : "ip"  
> },  
> "NodeName" : {  
> "type" : "keyword"  
> },  
> "NodeType" : {  
> "type" : "keyword"  
> },  
> "Uuid" : {  
> "type" : "keyword"  
> }  
> }  
> }  
> }  
> }  
> }

I am not sure whether this the problem but it might.  
First created index is 'register' with type 'register'. later there was some issues so we created other index with name 'noderegister' with type 'noderegister1' and moved all data from 'register' to noderegister and deleted the ole 'register' index and gave alias name 'noderegister' as 'register'.  
Above part is perfectly working. And when try to add data it is working but it is getting deleted immediately.

kindly check as huge data is getting deleted.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 6, 2018, 9:55am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/2 "2018-07-06T09:55:02Z")

</div>

There is nothing in elasticsearch which does that automatically.

So you have something or someone who is sending those requests to your cluster.

I hope it's not exposed publicly without any protection.

---

<div class="post-metadata">

**Author:** ![kish](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@kish](https://discuss.elastic.co/u/kish)\
**Post date:** [July 6, 2018, 10:09am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/3 "2018-07-06T10:09:09Z")

</div>

No, there is no such request coming. We are 100% sure.  
And also in indices output also we can see that docs are getting deleted less than 7-10sec.

is there any logs you need or where we can check?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 6, 2018, 10:36am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/4 "2018-07-06T10:36:30Z")

</div>

Try to remove the Network connection. If this is not happening anymore, then my theory is exact.

If it's still happening, run a ps -ef to see what is running locally.

---

<div class="post-metadata">

**Author:** ![kish](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@kish](https://discuss.elastic.co/u/kish)\
**Post date:** [July 6, 2018, 10:40am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/5 "2018-07-06T10:40:53Z")

</div>

> [@dadoonet](#):
>
> Try to remove the Network connection

Could you to explain it? you mean to say external connection? If then it is cluster node with no access to other network elements.  
If some process is deleting it then in at least we should be able to see in logs? i dont see any such thing.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 6, 2018, 11:05am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/6 "2018-07-06T11:05:15Z")

</div>

What is the output of

```
GET /_cat/nodes?v
GET /_cat/plugins?v
GET /_cat/indices?v
```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 6, 2018, 11:06am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/7 "2018-07-06T11:06:24Z")

</div>

I forgot to ask. Are you running with a old version of elasticsearch and using the now removed TTL feature?

---

<div class="post-metadata">

**Author:** ![kish](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@kish](https://discuss.elastic.co/u/kish)\
**Post date:** [July 6, 2018, 11:22am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/8 "2018-07-06T11:22:19Z")

</div>

please the details

```
root@cscfhcalloam002> curl -XGET 'localhost:9200'
{
  "name" : "cscfhcalloam002",
  "cluster_name" : "elastic-OAMUNIT",
  "cluster_uuid" : "hQBPen2kRNe8IlxOdbcTOg",
  "version" : {
    "number" : "5.1.1",
    "build_hash" : "5395e21",
    "build_date" : "2017-02-21T12:09:24.490Z",
    "build_snapshot" : true,
    "lucene_version" : "6.3.0"
  },
  "tagline" : "You Know, for Search"
}
root@cscfhcalloam002> curl -XGET 'localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open tdcorescaleio 6k2HYYU8SlOFbHJ1H-4lHQ 5 1 5 0 37.5kb 18.7kb
green open pmdatastore bPcWADZCSPSRXzGPMw9fXQ 5 1 0 0 1.3kb 708b
green open tdgmscaleio gaIu_X8fR_KLoSn7ZeNjrA 5 1 0 0 1.2kb 650b
green open hsscallpcounter I5jL7Y7IQ0KQ_kSlvm4dZw 5 1 0 0 1.3kb 708b
green open cscfcounter 9uMfrHm9R8ClBPSGTHBedQ 5 1 0 0 1.3kb 708b
green open noderegister Jddxe1pFRHSZeUZcMpvR0g 5 1 0 0 1.5kb 795b
green open fmdatastore gQ-1DzQ5SLG76CvTWtxoAA 5 1 0 0 1.2kb 650b
green open diameterlbcounter PVLzvzyBSp2w6HPIRbpexA 5 1 0 0 1.3kb 679b
green open cscfscaleio _aZ8bWFuSLuZzttodCEXKw 5 1 0 0 1.2kb 650b
green open tdgmcounter 60bcktJwSa6SFKyBk4kdnA 5 1 0 0 1.3kb 708b
green open tdcorecounter YPVTp0X5RC6N_y7ssIAeng 5 1 6 0 79.8kb 39.9kb
green open lbcounter owo9ZydeSEKMJMet_5J_dw 5 1 0 0 1.3kb 708b
green open queueio Wl-EzkwSTgWEjIjy4xMv_A 5 1 0 0 1.2kb 650b
root@cscfhcalloam002> curl -XGET 'localhost:9200/_cat/plugins?v'
name component version
root@cscfhcalloam002> curl -XGET 'localhost:9200/_cat/nodes?v'
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
2a00::108a 5 30 11 0.61 0.35 0.32 mdi - cscfhcalloam003
2a00::1089 8 33 40 2.19 2.22 2.15 mdi * cscfhcalloam002
2a00::1088 6 27 2 0.22 0.20 0.16 mdi - cscfhcalloam001
root@cscfhcalloam002>
```

---

<div class="post-metadata">

**Author:** ![kish](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@kish](https://discuss.elastic.co/u/kish)\
**Post date:** [July 6, 2018, 11:28am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/9 "2018-07-06T11:28:45Z")

</div>

few more details  
**root@cscfhcalloam002\> curl localhost:9200/\_cat/health?v**

> ```
> epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
> 1530876279 16:54:39 elastic-OAMUNIT green 3 3 130 65 0 0 0 0 - 100.0%
> 
> ```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 6, 2018, 11:47am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/10 "2018-07-06T11:47:10Z")

</div>

Please format your code, logs or configuration files using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and not the citation button. It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

This is the icon to use if you are not using markdown format:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7e6e239431ec2d71cbf1beef741f2e93e7cc762c.jpg)

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.  
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.  
Please update your post.

---

<div class="post-metadata">

**Author:** ![kish](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@kish](https://discuss.elastic.co/u/kish)\
**Post date:** [July 8, 2018, 2:52pm UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/11 "2018-07-08T14:52:34Z")

</div>

Any clue/help ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 8, 2018, 6:14pm UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/12 "2018-07-08T18:14:24Z")

</div>

Can you trace what are the network connections on each on your 3 nodes on port 9200?

---

<div class="post-metadata">

**Author:** ![kish](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@kish](https://discuss.elastic.co/u/kish)\
**Post date:** [July 11, 2018, 9:49am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/13 "2018-07-11T09:49:13Z")

</div>

At present i don't have the setup to share the details, will update once i get the setup.  
sorry for Inconvenience

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 8, 2018, 9:49am UTC](https://discuss.elastic.co/t/docs-are-getting-deleted-automatically/138879/14 "2018-08-08T09:49:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.