Retrieve the configured max_clause_count setting 8.x

mengjiann · May 21, 2024, 3:01am

Hi,

I understand that indices.query.bool.max_clause_count is deprecated in 8.x. The calculation is performed dynamically depending on the heap size and also the thread pool.

github.com

elastic/elasticsearch/blob/93fdfe5ae4d20c2aac56a5da0638aa207894a245/server/src/main/java/org/elasticsearch/search/SearchUtils.java#L18


      
          
          package org.elasticsearch.search;
          
          import org.elasticsearch.monitor.jvm.JvmStats;
          import org.elasticsearch.threadpool.ThreadPool;
          
          public final class SearchUtils {
          
              public static final int DEFAULT_MAX_CLAUSE_COUNT = 1024;
          
              public static int calculateMaxClauseValue(ThreadPool threadPool) {
                  int searchThreadPoolSize = threadPool.info(ThreadPool.Names.SEARCH).getMax();
                  long heapSize = JvmStats.jvmStats().getMem().getHeapMax().getMb();
                  return calculateMaxClauseValue(searchThreadPoolSize, heapSize);
              }
          
              static int calculateMaxClauseValue(long threadPoolSize, long heapInMb) {
                  if (threadPoolSize <= 0 || heapInMb <= 0) {
                      return DEFAULT_MAX_CLAUSE_COUNT;
                  }
                  // In a worst-case scenario, each clause may end up using up to 16k of memory

But is there is a way to retrieve it during runtime? I noticed there is setting returned from: curl localhost:9200/_nodes/settings. But, the value is always 6000 for the response from the endpoint:

"indices" : {
          "query" : {
            "bool" : {
              "max_clause_count" : "6000"
            }
          }
        },

mengjiann · May 21, 2024, 8:11am

It seems like the following result is obtained from _nodes/settings is because that the deprecated setting is configured in elasticsearch.yml file:

indices.query.bool.max_clause_count: 6000.

Removing the config will result in no config(max_clause_count) response for _nodes/settings

Carlos_D · May 22, 2024, 11:16am

Hi @mengjiann :

The setting is calculated at boot time and it will remain constant for a specific node.

Is there other behaviour that you would expect for this setting?

mengjiann · May 25, 2024, 5:20am

Hi Carlos,

Thanks for the reply. Yes, I'm aware that its calculated at boot time and remain constant after that. But, I'm looking for a way to retrieve the configured value after its boot up, eg. through the endpoint that I mentioned above (_nodes/settings) to get the value during runtime.

Is there any ways now?

Carlos_D · May 29, 2024, 11:54am

Got it - I don't believe there's a way to get this value at runtime.

What is your use case for checking it?

mengjiann · June 1, 2024, 3:16am

Noted. Is there plan to support the retrieval in the future?

Our web application allows users to add fields to be searched within the same index. There are some users who have added quite a number of fields and it results in the exceeding the max_clause_count error during the search on all the fields.

From the official documentation, the only way is to increase allocated memory so that the max_clause_count could be increased during runtime. Without knowing the calculated value during runtime, the allocation is done based on trial and error. Thus, knowing the runtime value would help us in the whole process.

The alternative could also be a redesign on our existing implementation for our web app.

Thanks again for checking.