# How to track why total number of docs is decreasing? **URL:** \ **Category:** Elasticsearch\ **Created:** [January 15, 2016, 3:13am UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297 "2016-01-15T03:13:30Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![lrhazi](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@lrhazi](https://discuss.elastic.co/u/lrhazi)\ **Post date:** [January 15, 2016, 3:13am UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/1 "2016-01-15T03:13:30Z")
I have lots of documents being indexed into an ES cluster, and I do not expect we are making use of TTLs in any index/mapping, nor do i expect we have any process that should ever delete documents... but the total number of documents, as reported by kopf plugin, is decreasing on regular basis... How can I track down why is it this happening? Is there some query logging maybe to enable? or could this number be decreasing for reasons other than "someone sent a delete document" request? Thanks a lot, Mohamed. ---
**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\ **Post date:** [January 15, 2016, 3:55am UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/2 "2016-01-15T03:55:52Z")
Does the issue replicate if you get a count of documents using the Elasticsearch APIs? If not, the issue is with the kopf plugin. If so, can you let us know and we'll probably need to gather more information from you? Let's a get a replication first solely within Elasticsearch though. ---
**Author:** ![lrhazi](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@lrhazi](https://discuss.elastic.co/u/lrhazi)\ **Post date:** [January 15, 2016, 4:02am UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/3 "2016-01-15T04:02:48Z")
Thanks. Like so: `[root@rap-es2 ~]# for i in $(seq 10);do !!;sleep 10;done for i in $(seq 10);do curl '[http://localhost:9200/\_cat/count?v](http://localhost:9200/_cat/count?v)';echo;sleep 10;done epoch timestamp count 1452830455 23:00:55 621610925 epoch timestamp count 1452830465 23:01:05 621610925 epoch timestamp count 1452830475 23:01:15 621610925 epoch timestamp count 1452830485 23:01:25 621610854 epoch timestamp count 1452830496 23:01:36 621610821 ^C [root@rap-es2 ~]# ` ---
**Author:** ![sandros](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandros/32/6348_2.png) [@sandros](https://discuss.elastic.co/u/sandros)\ **Post date:** [January 15, 2016, 9:30am UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/4 "2016-01-15T09:30:48Z")
Never used it but about checking the queries sent via slow.log? [https://www.elastic.co/guide/en/elasticsearch/reference/1.4/index-modules-slowlog.html](https://www.elastic.co/guide/en/elasticsearch/reference/1.4/index-modules-slowlog.html) ---
**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\ **Post date:** [January 15, 2016, 4:35pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/5 "2016-01-15T16:35:38Z")
> [@lrhazi](#): > > I do not expect we are making use of TTLs in any index/mapping Can you please verify that this is in fact the case? You should be able to do this with the [get mappings API](https://www.elastic.co/guide/en/elasticsearch/reference/2.x/indices-get-mapping.html): ```bash $ curl -XGET localhost:9200/_all/_mapping | grep _ttl ``` or the equivalent of your choice. ---
**Author:** ![lrhazi](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@lrhazi](https://discuss.elastic.co/u/lrhazi)\ **Post date:** [January 15, 2016, 4:57pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/6 "2016-01-15T16:57:22Z")
maybe I have a too old ES? `➜ ES rpm -qa| grep elas elasticsearch-1.2.4-1.noarch ➜ ES curl -s -XGET localhost:9200/_all/_mappings ;echo No handler found for uri [/_all/_mappings] and method [GET] ➜ ES` ---
**Author:** ![lrhazi](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@lrhazi](https://discuss.elastic.co/u/lrhazi)\ **Post date:** [January 15, 2016, 5:02pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/7 "2016-01-15T17:02:15Z")
I removed the s from mappings... Thanks! and thank you so much!! I do have \_ttl in all my indexes!!!! I can believe all the docs I lost.. Grrrrr!!! `"en_2015": { "mappings": { "doc": { "_source": { "enabled": false }, "_ttl": { "default": 7776000000, "enabled": true }, ...` ---
**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\ **Post date:** [January 15, 2016, 5:09pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/8 "2016-01-15T17:09:10Z")
> [@lrhazi](#): > > I do have \_ttl in all my indexes!!!! That makes _a lot_ more sense. I'm glad that we were able to track it down. 🙂 ---
**Author:** ![lrhazi](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@lrhazi](https://discuss.elastic.co/u/lrhazi)\ **Post date:** [January 15, 2016, 5:25pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/9 "2016-01-15T17:25:01Z")
One more question Jason.... How do I fix this now? This is defined in many (actually not all) of my indexes.... I guess I need to update the mapping for each... but would that also remove the ttl for already indexed documents? ---
**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\ **Post date:** [January 15, 2016, 6:52pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/10 "2016-01-15T18:52:05Z")
> [@lrhazi](#): > > How do I fix this now? This is defined in many (actually not all) of my indexes.... I guess I need to update the mapping for each... but would that also remove the ttl for already indexed documents? Correct, you need to update the mappings. Disabling the TTL on the index level will stop the purge thread from purging expired documents from the index, but it will not remove the `_ttl` field from the documents. ---
**Author:** ![lrhazi](https://avatars.discourse-cdn.com/v4/letter/l/e5b9ba/32.png) [@lrhazi](https://discuss.elastic.co/u/lrhazi)\ **Post date:** [January 15, 2016, 6:58pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/11 "2016-01-15T18:58:56Z")
Thanks... and disabling the purge is not available in 1.2 right? it is this setting: index.ttl.disable\_purge ? I see it in 1.7 docs as experimental: [https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-update-settings.html](https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-update-settings.html) ---
**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\ **Post date:** [January 15, 2016, 8:23pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/12 "2016-01-15T20:23:06Z")
> [@lrhazi](#): > > Thanks... and disabling the purge is not available in 1.2 right? I think that's been there since [#1791](https://github.com/elastic/elasticsearch/issues/1791) which is pre-1.2. But it's exactly because that flag is marked as experimental that I would update your mappings. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:23pm UTC](https://discuss.elastic.co/t/how-to-track-why-total-number-of-docs-is-decreasing/39297/13 "2017-07-05T23:23:59Z")