I'm analyzing how can I use ttl in my system to implement document
aging.
I've already configured indices.ttl.interval and indices.ttl.bulk_size
properties, and created the specific mapping for my index in:
I have now some questions:
1 - where can I see the possible time values? Can I use s for seconds
and m for minutes? d actually stands for days!
2 - this ttl will be coupled to each document, or if I change it, all
documents will reflect the change? And is there any other way to
change it besides the mapping update?
3 - why is my bulk_size property not working (is not being correctly
used to delete documents - is set to 100, but clears all my
documents)?
4 - to set the interval of tll process, can I use an regular
expression (like cron) or I'm limited to use time values?
the TTL defined in mapping will be applied to all indexed docs of type
"my_type" if you don't specify explicitly a TTL for each doc you index.
it is normal that all your expired docs are purged no matter what is
your bulk_size value. The bulk_size value just allow to configure how much
docs are expired at once by the purge process.
I deduce from your answer to 2), that a document once indexed wil a
ttl value, even if tll value is changed (by updating mapping), no
changes will be made to documents already indexed (unless we reindex
all documents), correct?
And, in 3), if I have 1000 documents, and have a bulk_size of 100 (and
for instance, ttl is running every minute and ttl value is set to 5m),
shouldn't I see deleted only 100 documents at a time? The problem is
that, is the situation above, all documents are deleted!
the TTL defined in mapping will be applied to all indexed docs of type
"my_type" if you don't specify explicitly a TTL for each doc you index.
it is normal that all your expired docs are purged no matter what is
your bulk_size value. The bulk_size value just allow to configure how much
docs are expired at once by the purge process.
changing the default TTL value in the mapping won't affect the TTL of
already indexed docs. If you want to update the TTL of an already indexed
document you can use the update API that has been recently added and doing
in your script something like this: ctx._ttl = "2d" (this is only available
in master branch for now and will be in next released version)
Nope. As it is currently implemented the purger is called, it collects
the 1000 expired documents, then executes 10 bulk requests of 100 delete
requests. So all expired documents are deleted.
changing the default TTL value in the mapping won't affect the TTL of
already indexed docs. If you want to update the TTL of an already indexed
document you can use the update API that has been recently added and doing
in your script something like this: ctx._ttl = "2d" (this is only available
in master branch for now and will be in next released version)
Nope. As it is currently implemented the purger is called, it collects
the 1000 expired documents, then executes 10 bulk requests of 100 delete
requests. So all expired documents are deleted.
Sorry to jump in on this - Benjamin, you say "changing the default TTL
in the mapping", and I've been trying to do exactly that without much
success on 0.18.6. Whatever I first use as the mapping seems to stick.
There's been some discussion of what can be updated for mappings but I
couldn't find any solid information. Is it possible to update the
default ttl?
changing the default TTL value in themappingwon't affect the TTL of
already indexed docs. If you want toupdatethe TTL of an already indexed
document you can use theupdateAPI that has been recently added and doing
in your script something like this: ctx._ttl = "2d" (this is only available
in master branch for now and will be in next released version)
Nope. As it is currently implemented the purger is called, it collects
the 1000 expired documents, then executes 10 bulk requests of 100 delete
requests. So all expired documents are deleted.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.