TTL doesn't work after installing Shield

Elastic Stack Elasticsearch
1

I've added a TTL field to my mapping: "_ttl": {"enabled": true,"default": "1m"}.
But Elasticsearch doesn't delete the documents after the given time. (waited +30min)

When i test the same thing without the shield plugin, everything works fine...

Setup:
Elasticsearch 1.5.2
Shield 1.2.1

2

Hi @JensP,

This is indeed a limitation of Shield that was missed in our documentation. We'll be updating the documentation to reflect this limitation shortly. We're thinking about how to address this limitation and the other limitations listed here.

Also, this issue highlights that no feedback was being provided to the user about the failed deletes in the logs. I'm also going to open an issue for that so the failures are not hidden in the future.

3

Hi @jaymode,

First of all thank you for the fast reply!

Is there an alternative to my problem? I really need shield for user authentication.
And documents must no longer be stored than 30 days.

Will this limitation be solved in the feature?

4

One alternative that I can think of requires some code outside of elasticsearch querying for a list of documents that are past their TTL (a range query less than or equal to the current time in milliseconds for the _ttl field) and then issuing deletes in a bulk request. This code would provide credentials that had a role in Shield with the appropriate permissions. This code would also run on a schedule.

What I described above is what elasticsearch does internally for TTL, but the addition is providing credentials.

We have been looking at all of the limitations of Shield and have been trying to think of the proper ways to solve them, so my hope is that we will be able to remove this limitation in the future. Unfortunately, I don't have a timeline right now.