The write bulk is stuck for a long time, causing the node load to rise and the cluster cannot be written

[linyy (yu hao)]

Elasticsearch version: 6.4.1
JVM version: openjdk 1.8.0_45
system: redhat 6.9

this task caused several nodes to load very high, and then the cluster could not write. Restart the node with high load to recover. Is there a solution? Thank you.

write tasks:
indices:data/write/bulk[s] Hun4IlhdR3mxhGPFBMM5fQ:1818928909 Hun4IlhdR3mxhGPFBMM5fQ:1818928875 transport 1539219616118 09:00:16 25.1m

{
"completed" : false,
"task" : {
"node" : "1Eb47iyTTYqO395zsINwlg",
"id" : 4372465,
"type" : "netty",
"action" : "indices:data/write/bulk[s]",
"status" : {
"phase" : "waiting_on_primary"
},
"description" : "requests[6], index[excc]",
"start_time_in_millis" : 1539219706425,
"running_time_in_nanos" : 1742763237947,
"cancellable" : false,
"parent_task_id" : "GfSLSR-qRqW7ZPVS0Crqyg:288914124",
"headers" : { }
}
}

node hot threads:
::: {1.1.1.1_hot}{mc0M2MimTBeLu8kyxvf5dw}{B0Sy-pRyTzul4TRfjlOcHA}{1.1.1.1}{1.1.1.1:9300}{ml.machine_memory=405669679104, ml.max_open_jobs=20,

xpack.installed=true, box_type=hot, ml.enabled=true}
Hot threads at 2018-10-09T03:31:09.720, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:

73.7% (368.7ms out of 500ms) cpu usage by thread 'elasticsearch[1.1.1.1_hot][write][T#27]'
5/10 snapshots sharing following 69 elements
java.math.BigInteger.multiply(BigInteger.java:1512)
java.math.BigInteger.multiplyToomCook3(BigInteger.java:1712)
java.math.BigInteger.multiply(BigInteger.java:1512)
java.math.BigInteger.multiplyToomCook3(BigInteger.java:1712)
java.math.BigInteger.multiply(BigInteger.java:1512)
java.math.BigInteger.multiplyToomCook3(BigInteger.java:1712)
java.math.BigInteger.multiply(BigInteger.java:1512)
java.math.BigInteger.multiplyToomCook3(BigInteger.java:1719)
java.math.BigInteger.multiply(BigInteger.java:1512)
java.math.BigInteger.multiplyToomCook3(BigInteger.java:1719)
java.math.BigInteger.multiply(BigInteger.java:1512)
java.math.BigInteger.multiplyToomCook3(BigInteger.java:1712)
java.math.BigInteger.multiply(BigInteger.java:1512)
java.math.BigInteger.pow(BigInteger.java:2302)
java.math.BigDecimal.bigTenToThe(BigDecimal.java:3543)
java.math.BigDecimal.bigMultiplyPowerTen(BigDecimal.java:3676)
java.math.BigDecimal.setScale(BigDecimal.java:2445)
java.math.BigDecimal.toBigInteger(BigDecimal.java:3025)
org.elasticsearch.common.xcontent.support.AbstractXContentParser.toLong(AbstractXContentParser.java:195)
org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:220)
org.elasticsearch.index.mapper.NumberFieldMapper$NumberType$7.parse(NumberFieldMapper.java:679)
org.elasticsearch.index.mapper.NumberFieldMapper$NumberType$7.parse(NumberFieldMapper.java:655)
org.elasticsearch.index.mapper.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:1010)

[dakrone (Lee Hinman)]

Do you have the document that is taking a long time to index?

You should be able to enable the indexing slowlog to log documents that are taking a long time to index:
https://www.elastic.co/guide/en/elasticsearch/reference/6.4/index-modules-slowlog.html#index-slow-log

It looks like it's taking a long time in the parsing of the document, so I am curious what it looks like.

[linyy (yu hao)]

Thank you for your reply, there are a few 1s slow index, but there is no hot threads on this node.

[linyy (yu hao)]

I modified this parameter to be changed to false

numeric_detection