Cache tunning for cardinality

Elastic Stack Elasticsearch
1

Hi I'm troubleshooting some wired case related to maybe cache ...
so when I'm using simple query by curl for the first shoot I got something like below

{"detail":"3,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]; nested exception is java.lang.RuntimeException: 3,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]","httpStatusCode":500,"instance":"/DE/product","title":"An unexpected error occurred while processing request","stackTraceElements":[{"classLoaderName":"app","methodName":"translateExceptionIfPossible","fileName":"ElasticsearchExceptionTranslator.java","lineNumber":75,"className":"org.springframework.data.elasticsearch.core.ElasticsearchExceptionTranslator","nativeMethod":false},{"classLoaderName":"app","methodName":"translateException","fileName":"ElasticsearchRestTemplate.java","

but for the second shoot I got the desired response. So it seems that elastic have a problem with cache or where should I look into?

 GET /_nodes/stats/indices/query_cache?human
	  
	  {
  "_nodes" : {
    "total" : 3,
    "successful" : 3,
    "failed" : 0
  },
  "cluster_name" : "elastic",
  "nodes" : {
    "iWa6kZJ5R_eyePwdxZq-Jw" : {
      "timestamp" : 1673520135682,
      "name" : "elastic-master-0",

      "roles" : [
        "data",
        "data_cold",
        "data_content",
        "data_frozen",
        "data_hot",
        "data_warm",
        "ingest",
        "master",
        "ml",
        "remote_cluster_client",
        "transform"
      ],
      "attributes" : {
        "ml.machine_memory" : "3221225472",
        "xpack.installed" : "true",
        "transform.node" : "true",
        "ml.max_open_jobs" : "512",
        "ml.max_jvm_size" : "1610612736"
      },
      "indices" : {
        "query_cache" : {
          "memory_size" : "1.7mb",
          "memory_size_in_bytes" : 1823137,
          "total_count" : 13946,
          "hit_count" : 276,
          "miss_count" : 13670,
          "cache_size" : 62,
          "cache_count" : 83,
          "evictions" : 21
        }
      }
    },
    "ejwSy1shQGmdqGTKrd39UQ" : {
      "timestamp" : 1673520135682,
      "name" : "elastic-master-2",

      "roles" : [
        "data",
        "data_cold",
        "data_content",
        "data_frozen",
        "data_hot",
        "data_warm",
        "ingest",
        "master",
        "ml",
        "remote_cluster_client",
        "transform"
      ],
      "attributes" : {
        "ml.machine_memory" : "3221225472",
        "ml.max_open_jobs" : "512",
        "xpack.installed" : "true",
        "ml.max_jvm_size" : "1610612736",
        "transform.node" : "true"
      },
      "indices" : {
        "query_cache" : {
          "memory_size" : "1.7mb",
          "memory_size_in_bytes" : 1850449,
          "total_count" : 81735131,
          "hit_count" : 2704122,
          "miss_count" : 79031009,
          "cache_size" : 32,
          "cache_count" : 64277,
          "evictions" : 64245
        }
      }
    },
    "-JV3TnmlTmiIFqbh-Kcp7A" : {
      "timestamp" : 1673520135683,
      "name" : "elastic-master-1",

      "roles" : [
        "data",
        "data_cold",
        "data_content",
        "data_frozen",
        "data_hot",
        "data_warm",
        "ingest",
        "master",
        "ml",
        "remote_cluster_client",
        "transform"
      ],
      "attributes" : {
        "ml.machine_memory" : "3221225472",
        "ml.max_open_jobs" : "512",
        "xpack.installed" : "true",
        "ml.max_jvm_size" : "1610612736",
        "transform.node" : "true"
      },
      "indices" : {
        "query_cache" : {
          "memory_size" : "1.8mb",
          "memory_size_in_bytes" : 1914209,
          "total_count" : 99455988,
          "hit_count" : 3507963,
          "miss_count" : 95948025,
          "cache_size" : 49,
          "cache_count" : 64847,
          "evictions" : 64798
        }
      }
    }
  }
}
2

It's the other way, the 2nd request is faster due to caching. This proofs that caching works.

Your error message originates from the spring framework, not from curl. It might be better to get in contact with the authors. Afaik this is not maintained by Elastic, but a 3rdparty client. The timeout 3s seems low, usually most clients use 30s. Apart from that the error handling looks wrong to me, a timeout should not be translated into a RuntimeException / 500.

3

Hi Hendrik thanks for getting some idea

What do You mean by ?

4

This corresponds to your error in the 1st post:

Your application runs into this timeout. The spring framework you are using translates this timeout into a runtime exception. This is in my opinion not ideal. However this is a problem of the client you are using. You could ask there.

Regarding the 3s timeout: Is this configured by you or a default?

5

I saw that in the def config they're using Spring Data Elasticsearch - Reference Documentation
.withSocketTimeout(Duration.ofSeconds(3))

6

if so we need to focus on cache tuning

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.