I am storing data in ES as follows:
- I have text and storing chunks of text as separate documents in ES.
- Each text has different permissions as to which user can view it.
I need most relevant passages and not text when searching so cannot store it as nested mapping since it will always give the relevance based on text and only the most relevant first chunk.
Considering this, what would be the optimum way in order to avoid permissions and other meta data duplicacy across each chunk without impacting performance or duplicate data is unavoidable.
Mention the approach at scale with a million text and each text having 10 chunks