Hi
I have a cluster on ElasticSearch Service (the managed service solution) in version 7.7. It has 2 data nodes (AWS.DATA.HIGHIO.I3 8GB) + a Master (AWS.MASTER.R4).
Am trying to enable Monitoring on it but I am facing a continuous error on one of the node where
(1) in the ES console, it shows that there is a disk allocation of zero on Node 1 (when Node 0 has a disk allocation of 111/250GB and a JVM pressure) - basically it seems the Node is not active though it shows in a status Green
(2) when I check the logs I get the following type of messages (see below) on this one node while the other node is not creating any issue
Does anyone has an idea about how to get rid of that issue?
[instance-0000000001] monitoring execution failed org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to add documents to export bulks at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.doAdd(ExportBulk.java:91) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.add(ExportBulk.java:52) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.Exporters.doExport(Exporters.java:236) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.Exporters.lambda$export$3(Exporters.java:215) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.delegateIfComplete(Exporters.java:319) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.onResponse(Exporters.java:298) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.onResponse(Exporters.java:269) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.local.LocalExporter.openBulk(LocalExporter.java:150) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.Exporters.wrapExportBulk(Exporters.java:201) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.exporter.Exporters.export(Exporters.java:213) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:261) [x-pack-monitoring-7.7.0.jar:7.7.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.7.0.jar:7.7.0] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:633) [elasticsearch-7.7.0.jar:7.7.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?] at java.lang.Thread.run(Thread.java:832) [?:?] Caused by: org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to add documents to export bulk [found-user-defined] at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.doAdd(LocalBulk.java:87) ~[?:?] at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.add(ExportBulk.java:52) ~[?:?] at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.doAdd(ExportBulk.java:88) ~[?:?] ... 18 more
