Hi all,
I installed an ES cluster over 3 servers for some research work:
(1) Server specs: 32 cores, Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz, 64G memory,
(2) 1 primary shard with 2 replica
(3) JVM with 15G memory allocated. No memory swapping
I'd like to check if any one met a similar problem:
(1) No concurrent index work
(2) The query cause 100% I/O usage on all 3 servers!
(3) while check "iostat". Almost all "i/o read", little "i/o write"
(4) No other work on the servers
(5) while check curl 'localhost:9200/_stats/query_cache?pretty&human', the cache hit rate is only 8--9%
(6) The runtime memory cost is about 200m ---1.5G
(7) elasticsearch.yml is configured:
threadpool.index.type: fixed
threadpool.index.queue_size: 1000
threadpool.bulk.queue_size: 1000
threadpool.search.queue_size: 3000
bootstrap.mlockall: true
index.cache.field.type: soft
index.cache.field.expire: 10m
index.cache.field.max_size: 50000
indices.cache.query.size: 5%
indices.cache.filter.size: 10gb
indices.fielddata.cache.size: 10gb
The queries have been simplified: search a keyword with a boolquery, without any filters, without any sort, without calculation any distances
The query time may take up to 10 seconds in peak hours. Normal time cost is about 50--500ms
Would great appreciate if anyone could share hints on this issue.
Best regards,
Ding