When deleting by query api the number of deleted_docs and index size_in_bytes does not decrease


(Rob Damiano) #1

Hi,

I've performed a delete by query and the number of deleted_docs and index
size_in_bytes does not decrease. I've also looked through the ES mailing
group and tried the recommended optimize, but I'm still not seeing expected
results. Here is my test case:

curl http://localhost:9200/apacheal/_stats/
http://localhost:9200/apacheal/_stats/

{"ok":true,"_shards":{"total":10,"successful":5,"failed":0},"_all":{"primaries":{"docs":{"count":
106407607,"deleted":809677},"store":{"size":"45.5gb","size_in_bytes":
48894062436
,"throttle_time":"7.8h","throttle_time_in_millis":28246172},"indexing":{"index_total":56322858,"index_time":"8.4h","index_time_in_millis":30412645,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}},"total":{"docs":{"count":106407607,"deleted":809677},"store":{"size":"45.5gb","size_in_bytes":48894062436,"throttle_time":"7.8h","throttle_time_in_millis":28246172},"indexing":{"index_total":56322858,"index_time":"8.4h","index_time_in_millis":30412645,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}}},"indices":{"apacheal":{"primaries":{"docs":{"count":106407607,"deleted":809677},"store":{"size":"45.5gb","size_in_bytes":48894062436,"throttle_time":"7.8h","throttle_time_in_millis":28246172},"indexing":{"index_total":56322858,"index_time":"8.4h","index_time_in_millis":30412645,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}},"total":{"docs":{"count":106407607,"deleted":809677},"store":{"size":"45.5gb","size_in_bytes":48894062436,"throttle_time":"7.8h","throttle_time_in_millis":28246172},"indexing":{"index_total":56322858,"index_time":"8.4h","index_time_in_millis":30412645,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}}}}}

curl -XDELETE http://localhost:9200/apacheal/events/_query
http://localhost:9200/apacheal/events/_query -d "{
"range":{"timestamp":{"lte":"2014-02-12T23:59:59-05:00"}} }"

{"ok":true,"_indices":{"apacheal":{"_shards":{"total":5,"successful":5,"failed":0}}}}

curl http:// http://logs.flyptech.com:9200/apacheal/_stats/localhost
http://localhost:9200/apacheal/_stats/:9200/apacheal/_stats/

{"ok":true,"_shards":{"total":10,"successful":5,"failed":0},"_all":{"primaries":{"docs":{"count":
100703300,"deleted":4961760},"store":{"size":"45.5gb","size_in_bytes":
48896498586
,"throttle_time":"7.8h","throttle_time_in_millis":28246326},"indexing":{"index_total":56324061,"index_time":"8.4h","index_time_in_millis":30413483,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}},"total":{"docs":{"count":100703300,"deleted":4961760},"store":{"size":"45.5gb","size_in_bytes":48896498586,"throttle_time":"7.8h","throttle_time_in_millis":28246326},"indexing":{"index_total":56324061,"index_time":"8.4h","index_time_in_millis":30413483,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}}},"indices":{"apacheal":{"primaries":{"docs":{"count":100703300,"deleted":4961760},"store":{"size":"45.5gb","size_in_bytes":48896498586,"throttle_time":"7.8h","throttle_time_in_millis":28246326},"indexing":{"index_total":56324061,"index_time":"8.4h","index_time_in_millis":30413483,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}},"total":{"docs":{"count":100703300,"deleted":4961760},"store":{"size":"45.5gb","size_in_bytes":48896498586,"throttle_time":"7.8h","throttle_time_in_millis":28246326},"indexing":{"index_total":56324061,"index_time":"8.4h","index_time_in_millis":30413483,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}}}}}

RAN OPTIMIZE THROUGH UI

curl http:// http://logs.flyptech.com:9200/apacheal/_stats/localhost
http://localhost:9200/apacheal/_stats/
:9200/apacheal/_stats/
http://logs.flyptech.com:9200/apacheal/_stats/

{"ok":true,"_shards":{"total":10,"successful":5,"failed":0},"_all":{"primaries":{"docs":{"count":
100705106,"deleted":4961760},"store":{"size":"44.8gb","size_in_bytes":
48159890359
,"throttle_time":"7.8h","throttle_time_in_millis":28246687},"indexing":{"index_total":56325867,"index_time":"8.4h","index_time_in_millis":30417576,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}},"total":{"docs":{"count":100705106,"deleted":4961760},"store":{"size":"44.8gb","size_in_bytes":48159890359,"throttle_time":"7.8h","throttle_time_in_millis":28246687},"indexing":{"index_total":56325867,"index_time":"8.4h","index_time_in_millis":30417576,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}}},"indices":{"apacheal":{"primaries":{"docs":{"count":100705106,"deleted":4961760},"store":{"size":"44.8gb","size_in_bytes":48159890359,"throttle_time":"7.8h","throttle_time_in_millis":28246687},"indexing":{"index_total":56325867,"index_time":"8.4h","index_time_in_millis":30417576,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}},"total":{"docs":{"count":100705106,"deleted":4961760},"store":{"size":"44.8gb","size_in_bytes":48159890359,"throttle_time":"7.8h","throttle_time_in_millis":28246687},"indexing":{"index_total":56325867,"index_time":"8.4h","index_time_in_millis":30417576,"index_current":0,"delete_total":0,"delete_time":"0s","delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"get_time":"0s","time_in_millis":0,"exists_total":0,"exists_time":"0s","exists_time_in_millis":0,"missing_total":0,"missing_time":"0s","missing_time_in_millis":0,"current":0},"
search
":{"open_contexts":0,"query_total":2020,"query_time":"2.5h","query_time_in_millis":9351789,"query_current":0,"fetch_total":1536,"fetch_time":"4.4m","fetch_time_in_millis":267579,"fetch_current":0}}}}}

When I deleted, the document count went down by ~5.7 million records, which
was expected, but the deleted only increase by ~ 4.2 million, which is not
exactly expected. The size_in_bytes only decrease by ~ 2.4 MB, and then
after the optimize only decrease by ~ 700 MB. This is not what I was
expecting at all. Each of the records is about 500 bytes, so 500 bytes *
5.7 million records = 2.65 GB. Why is the deleted documents and size in
bytes not decreasing significantly?

Thanks for your help,

Rob

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1bc61dd1-26ff-4e0a-89a7-c5f2ba95cc10%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Binh Ly-2) #2

Curious, when you ran optimize, what did you pass to the max_num_segments
parameter?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1c8980ca-7305-4a2e-a1b5-40cc104d3616%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Rob Damiano) #3

I passed 1.

Rob

On Thu, Mar 6, 2014 at 6:13 PM, Binh Ly binhly_es@yahoo.com wrote:

Curious, when you ran optimize, what did you pass to the max_num_segments
parameter?

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/-3zp0biP4dQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1c8980ca-7305-4a2e-a1b5-40cc104d3616%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/1c8980ca-7305-4a2e-a1b5-40cc104d3616%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

*ROBERT DAMIANO *

Senior Developer

T: 888 694 2946

*E: *rob.damiano@uberflip.com

W: uberflip.com

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BUUMJjbcSwd_%2Bnwee3qB3U4ZOj8DLJ5U_N61x9S0dbP9K1XfQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #4