How to reduce elasticsearch memory usage

[lovelly]

I use the bulk api to insert a lot of data. Elasticsearch will quickly consume memory and then crash.

elasticsearcg log :
[2019-11-15T14:13:00,235][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1287] overhead, spent [286ms] collecting in the last [1s]
[2019-11-15T14:13:05,273][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1292] overhead, spent [280ms] collecting in the last [1s]
[2019-11-15T14:13:06,274][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1293] overhead, spent [319ms] collecting in the last [1s]
[2019-11-15T14:13:09,389][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1296] overhead, spent [282ms] collecting in the last [1.1s]
[2019-11-15T14:13:30,889][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1317] overhead, spent [262ms] collecting in the last [1s]
[2019-11-15T14:13:35,892][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1322] overhead, spent [280ms] collecting in the last [1s]
[2019-11-15T14:13:36,960][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1323] overhead, spent [271ms] collecting in the last [1s]
[2019-11-15T14:13:41,098][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1327] overhead, spent [524ms] collecting in the last [1.1s]
[2019-11-15T14:13:47,179][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1333] overhead, spent [274ms] collecting in the last [1s]
[2019-11-15T14:13:55,217][INFO ][o.e.m.j.JvmGcMonitorService] [es1] [gc][1341] overhead, spent [295ms] collecting in the last [1s]

Segments memory:
curl "localhost:9200/_cat/nodes?v&h=name,port,sm"
name port sm
es1 9301 5.8mb
es2 9302 8mb

machine memory
free -mh
total used free shared buff/cache available
Mem: 31G 22G 746M 186M 8.2G 8.3G
Swap: 0B 0B 0B

[philippkahr]

How many nodes do you have? How fast are your disks? how big are the disks attached? You can turn on x.pack.monitoring in kibana in the monitoring tab, than you would see the internals of elastic and see how much memory a specific index is taking.