Mass TTL

Elastic Stack Elasticsearch
1

I have an issue when I bulk insert a large number (~17k) of documents with
the same TTL. It seems to delete some (~2k) of them, but the rest seem to
just sit there forever. It then attempts to delete these documents again
every TTL interval and I get errors for them in the error log. Here is a
sample of the error I'm getting.

[2012-01-03 15:29:05,225][DEBUG][action.bulk ] [Nelson, Greer
Grant] [autocomplete_test][7] failed to bulk item (delete) delete
{[autocomplete_test][searchSuggestion][c550997558871ca5729691e1e3d9dd51]}
org.elasticsearch.index.engine.VersionConflictEngineException:
[autocomplete_test][7]
[searchSuggestion][c550997558871ca5729691e1e3d9dd51]: version conflict,
current [-1], required [1]
at
org.elasticsearch.index.engine.robin.RobinEngine.innerDelete(RobinEngine.java:646)
at
org.elasticsearch.index.engine.robin.RobinEngine.delete(RobinEngine.java:597)
at
org.elasticsearch.index.shard.service.InternalIndexShard.delete(InternalIndexShard.java:336)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:178)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:487)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:400)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

2

That failure is actually no longer logged in master (version conflict
failures on DEBUG level). But, it might help us here. Are you suggeting
that the document logged in your case is still there in the index
(c550997558871ca5729691e1e3d9dd51)? Cause that version conflict means
trying to delete a document that is already deleted.

On Wed, Jan 4, 2012 at 1:36 AM, James Bathgate julesbravo@gmail.com wrote:

I have an issue when I bulk insert a large number (~17k) of documents with
the same TTL. It seems to delete some (~2k) of them, but the rest seem to
just sit there forever. It then attempts to delete these documents again
every TTL interval and I get errors for them in the error log. Here is a
sample of the error I'm getting.

[2012-01-03 15:29:05,225][DEBUG][action.bulk ] [Nelson,
Greer Grant] [autocomplete_test][7] failed to bulk item (delete) delete
{[autocomplete_test][searchSuggestion][c550997558871ca5729691e1e3d9dd51]}
org.elasticsearch.index.engine.VersionConflictEngineException:
[autocomplete_test][7]
[searchSuggestion][c550997558871ca5729691e1e3d9dd51]: version conflict,
current [-1], required [1]
at
org.elasticsearch.index.engine.robin.RobinEngine.innerDelete(RobinEngine.java:646)
at
org.elasticsearch.index.engine.robin.RobinEngine.delete(RobinEngine.java:597)
at
org.elasticsearch.index.shard.service.InternalIndexShard.delete(InternalIndexShard.java:336)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:178)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:487)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:400)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

3

Shay,

The document logged is still in the index.

Here is a log entry:
org.elasticsearch.index.engine.VersionConflictEngineException:
[autocomplete_test][6]
[searchSuggestion][a3f421daecd76189d6ff5fcf0992e9b6]: version conflict,
current [-1], required [1]
[2012-01-04 08:33:18,987][DEBUG][action.bulk ] [Nelson, Greer
Grant] [autocomplete_test][6] failed to bulk item (delete) delete
{[autocomplete_test][searchSuggestion][a3f421daecd76189d6ff5fcf0992e9b6]}

Here is a query hitting the server and it's result:
jbathgate@crimson:~/Projects/es$ curl -XPOST
'http://localhost:9200/autocomplete_test/searchSuggestion/_search' -d
'{"size": 1, "query":{"match_all":{}}}'
{"took":2,"timed_out":false,"_shards":{"total":10,"successful":10,"failed":0},"hits":{"total":15656,"max_score":1.0,"hits":[{"_index":"autocomplete_test","_type":"searchSuggestion","_id":"a3f421daecd76189d6ff5fcf0992e9b6","_score":1.0,
"_source" :
{"_id":"a3f421daecd76189d6ff5fcf0992e9b6","siteId":"vw50sm","searchQuery":"pro
circuit","searchQueryAutocompleteExact":"pro
circuit","popularity":0.60427935600034}}]}}

As you can see that document definitely still exists.