Sometimes index docs very slow

Elastic Stack Elasticsearch
1

Today,i find my es cluster index docs very slow and precarious.

I added some log to calculate how long to call the bulk api.

public void execute() throws Exception { logger.info("begin call es bulk api"); long startTime=System.currentTimeMillis(); try { BulkResponse bulkResponse = bulkRequestBuilder.execute().actionGet(); if (bulkResponse.hasFailures()) { throw new EventDeliveryException(bulkResponse.buildFailureMessage()); } } finally { bulkRequestBuilder = client.prepareBulk(); } logger.info("end es bulk api,time:"+(System.currentTimeMillis() -startTime)); }

From the log.
end es bulk api,time:100 end es bulk api,time:1121 end es bulk api,time:56785 end es bulk api,time:16201 end es bulk api,time:401 end es bulk api,time:87543
I find es sometimes index docs very slow.

I use the api curl 'localhost:9200/_cat/thread_poolv&h=id,host,bulk.size,bulk.active,bulk.rejected,bulk.completed,bulk.queueSize,bulk.queue' and find the active.bulk and buld.queue are all less 2.

I use the command top and iotop and find the cpu usage and disk io are all very small.

How to find out where the problem is?
Thank you

2

Are your bulk indexing requests always the same number of documents, size of documents? If that varies then indexing time would also vary.

Try following the suggestions from this blog post?: https://www.elastic.co/blog/performance-indexing-2-0

If it's still randomly slow, can you post hot threads during a slow bulk request?

Mike