Parent-child join and time-based indexes

Elastic Stack Elasticsearch
1

Here is a short description of my problem, we have:

*) One to many relation with lots of documents (hundreds of thousands) on the one side, and a large size on the many side (a couple of thousand per document on the one side).
*) Many side is time-based so we want to TTL it, I believe best practices involve time-based indexing for the many side.
*) One to many would be suitable for the parent-child relationship, one side gets updated often, many side is write-once.

Any ideas on how to allow join queries here?

*) If we go time-based indexing on the many side we will no longer support the parent-child relationship correct?
*) If we do not go time-based indexing on the many side we will have to pay the price of huge indexes and expensive TTL purging? Any efficient way to do it?
*) If we do manual joins, we might bump into the maximum number of ids issue here again, what is that limit?

BTW: I am experimenting with Elastic 5.0.0 alpha1 right now.

Any help appreciated, thanks,

Francisco.

2

There is no ideal solution here.

You could consider including the ID of the "parent" in the "child" event that gets created in the time based index, then you can do the "join" externally.

3

Thanks for the quick answer. Yup, I might have to go for something like that...