Questions about ES Performance optimization, can anyone help me?

Elastic Stack Elasticsearch
1

Hello, I'm testing indexing performance about es.

under these conditions :

  1. ES 5.4.0 , a single node cluster.

  2. -Xmx17179869184 -XX:MaxDirectMemorySize=17179869184

  3. use _bulk api, 500docs per request.

  4. 20 clients of jmeter.

  5. path.data:
    /data2/zdh/elasticsearch/data,/data3/zdh/elasticsearch/data,/data4/zdh/elasticsearch/data,/data5/zdh/elasticsearch/data,/data6/zdh/elasticsearch/data,/data7/zdh/elasticsearch/data,/data8/zdh/elasticsearch/data,/data9/zdh/elasticsearch/data,/data10/zdh/elasticsearch/data,/data11/zdh/elasticsearch/data

  6. index named test with 12 shards.

  7. disk infos.

  8. memory info

  9. cpu about 48 processors, top info:

results:
I use Jconsole to watch es, and I found lots of threads blocked at TranslogWriter.add()

866×396 56.7 KB

and I found the memory used by off-heap is only 1G:

882×578 46.7 KB

Is that OK?

and the IO:

751×229 7.74 KB

at one moment , just one or two hard disk are busy, the others are idle. I do't know why.

thread pool about bulk:

"thread_pool" : {
"bulk" : {
"threads" : 32,
"queue" : 88,
"active" : 32,
"rejected" : 325,
"largest" : 32,
"completed" : 165387
}

memery usage of jvm: 37%

I think the disk is not been fully used , Is there anything I can do to optimize the performance? thank you.

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.