I am using Elasticsearch 2.3.3 and I found out that _ttl (time to live) mapping was deprecated. I am wondering if there is any alternative mapping for automatic deletion of documents. Any comments and suggestions would be helpful.
No mapping, but usually we recommend using time based indexes. Like make an index every day or week or something and remove it when you don't need it. The trouble with ttl is that removing documents is way way more costly than removing indexes.
You can certainly add a timestamp field to the document (not _timestamp, that too is deprecated) and do a delete-by-query with a range of timestamps. And that is a fine thing to do on a small index, but it is only fine if the index is small.
As per your suggestion, I used delete-by-query but I am getting following exception
{"error":{"root_cause":[{"type":"parse_exception","reason":"Failed to derive xcontent"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query"
,"grouped":true,"failed_shards":[{"shard":0,"index":"accounts_test_power_user","node":"2V96EnCPRJevhWIUs-yeRA","reason":{"type":"parse_exception","reason":"Failed to derive xc
ontent"}}]},"status":400}
I created "accounts_test_power_user" index with mappings(expiration_date is one field) and I gave mapping for expiration_date
"expiration_date" : {
"type" : "date",
"format" : "MM/dd/yyyy"
}
For delete-by-query, I used below curl request
curl --XDELETE http://localhost:9200/accounts_test_power_user/external/_query -d'{"filter":{"range":{"expiration_date":{"lte":"12/19/2016","format":"MM/dd/yyyy"}}}}'
Please suggest me if I can use this approach to delete the data.