Terms is not fetching the results if total number of terms is more

Elastic Stack Elasticsearch
1

I am tring to filter the data using Elasticsearch terms query.I am passing morethan 300000 inputs to the terms query.Finally it ended up by showing the memory issue.

How can i resolve this issue.
Please help.

Thanks in Advance

2

afaik TermsQuery are limited to 1024 terms. Loop over your terms divided by group of 1024, and make a multirequest.

3

ok.

Is it right decision by divinding the total 300000 terms spliting and excuting there will be multiple queries triggred to elasticsearch.

Does by increasing the count index.query.bool.max_clause_count: 1000240 does it help ful any way

thanks

4

Thank you very much,It worked.

But my concern is there would be too many queries to elasticsearch.Is it fine?
It cannot impact any performance i belive ?
please suggest.

Thanks .

5

Hi,

Yes, It may, because 300k / 1024 = 292 queries running.
You realy need to send 300k of values ? it cannot be a range for exemple ?

bye,
Xavier

6

Thanks For reply.

No we cannot use Range query.The reason is i have to fetch for the selected data only .I need to fetch.So we are using terms query.

That is the reason we using terms query.does it may have any performance issue.

Thanks
Venkat

7

I cannot tell you if yes or no, it depends of your dataset, etc..., it's just going to make lot of queries.

bye,
Xavier

8

Thanks for the reply.

Can i increase the Default max clause count by using the below config change in the elasticsearch cofig file.
index.query.bool.max_clause_count: 10240.

By this i can reduce my queries by 10times i.e to 292 queries to 29 or 30 queries.
suggest is that it can be good approach?

Thanks
Venkat

9

Hi,

I understand, but I cannot answer to this question.
cc @dadoonet @warkolm any idea ?

10

I dunno. @jpountz can hopefully help.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.