Hello,
Could anyone please tell me what kind of error is this
:
It is in my elasticsearch log /var/log/elasticsearch/elktest01.log.
[2024-02-08T09:03:07,475][WARN ][o.e.x.m.MonitoringService] [testdata01] monitoring execution failed
org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulks
at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$doFlush$0(ExportBulk.java:110) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:144) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.lambda$doFlush$1(LocalBulk.java:121) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:144) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.support.ContextPreservingActionListener.onFailure(ContextPreservingActionListener.java:38) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:97) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.ActionListener$Delegating.onFailure(ActionListener.java:66) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.ActionListener$RunBeforeActionListener.onFailure(ActionListener.java:399) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.bulk.TransportBulkAction.doInternalExecute(TransportBulkAction.java:247) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.bulk.TransportBulkAction$1.doRun(TransportBulkAction.java:192) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:200) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:91) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:186) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:161) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:82) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:95) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:73) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.client.support.AbstractClient.bulk(AbstractClient.java:492) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:127) [x-pack-core-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.local.LocalBulk.doFlush(LocalBulk.java:111) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.flush(ExportBulk.java:64) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$doFlush$1(ExportBulk.java:108) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:117) [x-pack-core-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.doFlush(ExportBulk.java:127) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.ExportBulk.flush(ExportBulk.java:64) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.Exporters.doExport(Exporters.java:296) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.Exporters.lambda$export$3(Exporters.java:271) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.delegateIfComplete(Exporters.java:380) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.onResponse(Exporters.java:359) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.Exporters$AccumulatingExportBulkActionListener.onResponse(Exporters.java:326) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.local.LocalExporter.openBulk(LocalExporter.java:246) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.Exporters.wrapExportBulk(Exporters.java:256) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.exporter.Exporters.export(Exporters.java:269) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:284) [x-pack-monitoring-7.17.17.jar:7.17.17]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.17.17.jar:7.17.17]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:317) [?:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:718) [elasticsearch-7.17.17.jar:7.17.17]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
at java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulk [default_local]
... 42 more
Caused by: java.lang.IllegalStateException: There are no ingest nodes in this cluster, unable to forward request to an ingest node.
at org.elasticsearch.action.ingest.IngestActionForwarder.randomIngestNode(IngestActionForwarder.java:52) ~[elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.ingest.IngestActionForwarder.forwardIngestRequest(IngestActionForwarder.java:42) ~[elasticsearch-7.17.17.jar:7.17.17]
at org.elasticsearch.action.bulk.TransportBulkAction.doInternalExecute(TransportBulkAction.java:244) ~[elasticsearch-7.17.17.jar:7.17.17]
... 35 more
I installed metricbeat to monitor cluster health.
it is working fine for some time like this :
And after while i got this error:
When i change node.roles: [ node,master,ingest ] , error dissapear and also error [Unable to find the cluster in the selected time range] dissapear. than i have monitoring ok. like in the 1st picture, when i remove ingest role. it will happen again after while.
