Elasticsearch docs purging

Elastic Stack Elasticsearch
1

Hi,

Is there a ES function that can replace the epoch time? Currently we purge the doc according to ttl for past 35 days. is there a better way to handle it?

{"size":0,
"query": {
"bool": {
"must": [
{
"exists" : {
"field" : "hubtimestamp"
}
},
{
"script": {
"script": {
"lang": "expression",
"source": "((doc["ttl_i"].value ==0 ? 35 :doc["ttl_i"].value) 2460601000)+ doc["hubtimestamp"].value <=datenow",
"params": {
"datenow": {$EPOCH_TIME} <<<<==== need ES function to populate the value.
}
}
}
}
]
}
}
}

2

Hey,

From the outside it seems to me as if you are trying to solve the wrong problem. How about having time-based indices (weekly, daily, whatever), which will allow you to simply delete a whole index after 35 days. This is much easier and much cheaper to do, than trying to delete documents based on a query.

If you have per document TTLs you could index those documents based on their TTL and basically do the same.

If this is not an option, why not add a fixed TTL timestamp on index time, so that deletion becomes easier by just specifying a date range, this way you would not need a script?

--Alex

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.