Error Activating Self-Monitoring in ElasticSearch 7.17.3

Elastic Stack Elasticsearch
1

Hi, I have problem with Self-Monitoring in Elasticsearch 7.17.3.
It worked correctly, but I changed Stting parameters with
PUT 'http: // server:9200 /_settings
{
"index": {
"final_pipeline": "geoip-info"
}
}
My foult :frowning_face:

After I trying to fix it with:
PUT http :// server:9200 /_settings
{
"index": {
"final_pipeline": null
}
}

Filebeat/Heartbeat was restored, but self-monitoring is still not available.

I found one topic with issue like my and solution was to remove index template (monitoring-es), but it's a system template and I afraid to delete it )
How can i fix this issue? How can I find what is problem witn Self Monitoring?

Elasticsearch 7.17.3 cluster with 3 data-nodes and 1 Kibana

2

What do your Elasticsearch logs show?

3

[2022-05-13T00:00:00,374][WARN ][o.e.x.m.MonitoringService] [tb-elk001.bee.mobitel.local] monitoring execution failed
org.Elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulks
at org.Elasticsearch.xpack.monitoring.exporter.ExportBulk$Compound.lambda$doFlush$0(ExportBulk.java:110) [x-pack-monitoring-7.17.3.jar:7.17.3]
at org.Elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:144) [Elasticsearch-7.17.3.jar:7.17.3]
etc...
Caused by: org.Elasticsearch.xpack.monitoring.exporter.ExportException: failed to flush export bulk [default_local]
... 101 more
Caused by: java.lang.IllegalStateException: There are no ingest nodes in this cluster, unable to forward request to an ingest node.
at org.Elasticsearch.action.ingest.IngestActionForwarder.randomIngestNode(IngestActionForwarder.java:52) ~[Elasticsearch-7.17.3.jar:7.17.3]
at org.Elasticsearch.action.ingest.IngestActionForwarder.forwardIngestRequest(IngestActionForwarder.java:42) ~[Elasticsearch-7.17.3.jar:7.17.3]
at org.Elasticsearch.action.bulk.TransportBulkAction.doInternalExecute(TransportBulkAction.java:244) ~[Elasticsearch-7.17.3.jar:7.17.3]
... 92 more

4

This would be worth checking. You can use the _cat/nodes?v API to check what roles your nodes have enabled.

5

image

6

Please don't post pictures of text, logs or code. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them :slight_smile:

7

Ok :slight_smile:
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
x.x.x.223 58 98 0 0.16 0.18 0.12 dm - elk001
x.x.x.115 17 82 0 0.00 0.01 0.05 lr - kibana
x.x.x.224 36 97 0 0.10 0.04 0.05 dm * elk002
x.x.x.225 11 99 0 0.03 0.04 0.05 dm - elk003

8

Thanks.

Have you explicitly set the node roles in the configs?

9

node.roles: [ master, data ] - elk001
node.roles: [ data, master ] - elk002
node.roles: [ data, master ] - elk003

kibana

node.name: kibana
node.master: false
node.data: false
node.ingest: false

10

Problem was solved after setting up metricbeat monitoring.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.