Hello,
I am using a Lucene query for building dynamic search queries. Let's assume a sample search will take the following format:
Blockquote
(Gender:"Female" AND AgeRange:[17 TO 24] AND Sports:"YES" AND State:"TX") OR (Gender:"Female" AND AgeRange:[17 TO 24] AND Sports:"YES" AND State:"CA")
A user will always want to exclude results they have received on previous search with identical parameters. I am able to achieve suppressing the previous results by constructing the following query
Blockquote
(Gender:"Female" AND AgeRange:[17 TO 24] AND Sports:"YES" AND State:"TX") OR (Gender:"Female" AND AgeRange:[17 TO 24] AND Sports:"YES" AND State:"CA") AND (-id:10 OR -id:11 OR ...)
The query can end up with > 100,000 terms. The large queries are crashing my client node and do not perform well. Do you have a suggestions on a better way of achieving the aforementioned goals.
My current setup consist of the following running Elasticsearch release 7.9:
1 Client Node: 4 core 16gb Ram
3 Master Nodes: 4 core 8gb Ram
3 Data Nodes: 16 core 64 gb Ram 2TB hard drive space
The cluster is not indexing any data on an ongoing basis other than Kibana data. I have the index refresh rate set to -1. I currently load data on a monthly basis so the cluster is relatively static. Search is the primary job of the cluster.
Thank You In Advance