Elastic search: Slow performance on large data set

It is hard to analytically try to determine whether your current model will or will not perform better than e.g. the possible alternative I described earlier. If you simplify and make indexing more performant you may at the same time make querying more expensive and vice versa. How the balance swings may depend on the data model, cluster configuration, hardware limitations as well as distribution in time of indexing and query load. Running realistic benchmarks is therefore often the only way to determine this.

It may be that someone else here in the forum has implemented something similar to what you are looking to do and can provide better guidance based on practical experience, but I would need to resort to benchmarking to be sure.