Hi @rashmi thanks for the reply.
I'm sorry i cant able to find my kibana logs.
Have a look at my elasticsearch logs:
[2019-06-28T21:39:29,154][INFO ][o.e.c.m.MetaDataIndexTemplateService] [YWG9VCA] adding template [.management-beats] for index patterns [.mangement-beats]
[2019-06-28T21:39:50,002][WARN ][o.e.c.r.a.DiskThresholdMonitor] [YWG9VCA] flood stage disk watermark [95%] exceeded on [YWG9VCAsQwm7rYvNT57S8A][YWG9VCA][/var/lib/elasticsearch/nodes/0] free: 28gb[3.1%] , all indices on this node will be marked read-only
[2019-06-28T21:40:20,005][WARN ][o.e.c.r.a.DiskThresholdMonitor] [YWG9VCA] flood stage disk watermark [95%] exceeded on [YWG9VCAsQwm7rYvNT57S8A][YWG9VCA][/var/lib/elasticsearch/nodes/0] free: 28gb[3.1%] , all indices on this node will be marked read-only
[2019-06-28T21:40:50,006][WARN ][o.e.c.r.a.DiskThresholdMonitor] [YWG9VCA] flood stage disk watermark [95%] exceeded on [YWG9VCAsQwm7rYvNT57S8A][YWG9VCA][/var/lib/elasticsearch/nodes/0] free: 28gb[3.1%] , all indices on this node will be marked read-only
[2019-06-28T21:41:20,011][WARN ][o.e.c.r.a.DiskThresholdMonitor] [YWG9VCA] flood stage disk watermark [95%] exceeded on [YWG9VCAsQwm7rYvNT57S8A][YWG9VCA][/var/lib/elasticsearch/nodes/0] free: 28gb[3.1%] , all indices on this node will be marked read-only
[2019-06-28T21:41:50,016][WARN ][o.e.c.r.a.DiskThresholdMonitor] [YWG9VCA] flood stage disk watermark [95%] exceeded on [YWG9VCAsQwm7rYvNT57S8A][YWG9VCA][/var/lib/elasticsearch/nodes/0] free: 28gb[3.1%] , all indices on this node will be marked read-only
oh as the stack trace indicates , you have reached high watermark. cluster.routing.allocation.disk.watermark.flood_stage this setting defaults to 95%, meaning that Elasticsearch enforces a read-only index block ( index.blocks.read_only_allow_delete ) on every index that has one or more shards allocated on the node that has at least one disk exceeding the flood stage. This is a last resort to prevent nodes from running out of disk space. The index block must be released manually once there is enough disk space available to allow indexing operations to continue. Disk-based shard allocation | Elasticsearch Guide [8.11] | Elastic
May be delete the .kibana index and restart ? There may be other things which might be hogging your network as well.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.