Has anyone had to tackle effective dating?


(Roly Vicaria) #1

We're using MongoDB as our primary data store and using a river plugin to
get data flowing into ES. Question we're facing now is how to tackle
effective dating. We figure that we could ignore it and only push current
docs into ES. This would force us to drop our indices every night (midnight
sounds good) and rebuild them with all the currently effective docs. I'm
curious if ES has any mechanism that could assist. I saw that there are
facilities for timestamps, ttl, and versioning. It feels like those
features could help solve the problem on the ES side, but the solution
isn't obvious to me. Anyone come across this problem?


(vineeth mohan) #2

"Dating" questions asked in ES forum .... that is something new :smiley:

~Vineeth

On Fri, Jan 13, 2012 at 3:21 AM, Roly Vicaria rolyv19@gmail.com wrote:

We're using MongoDB as our primary data store and using a river plugin to
get data flowing into ES. Question we're facing now is how to tackle
effective dating. We figure that we could ignore it and only push current
docs into ES. This would force us to drop our indices every night (midnight
sounds good) and rebuild them with all the currently effective docs. I'm
curious if ES has any mechanism that could assist. I saw that there are
facilities for timestamps, ttl, and versioning. It feels like those
features could help solve the problem on the ES side, but the solution
isn't obvious to me. Anyone come across this problem?


(Karussell) #3

"Dating" questions asked in ES forum .... that is something new :smiley:

Personal dating we cannot solve here :wink:

Regarding your software problem, yes check out ttl and also how to use
a rolling index:

Peter.


(system) #4