Graphs not showing

I have taken reboot of of my kibana cluster, post reboot the graphs are not showing neither for 15mins or any orher timeline.Earlier it was showing exceptions of

[2] total shards, but this cluster currently has [3902]/[3000] maximum shards open;

but post deleteing the old indexes, that exceptions gone but still graphs not showing.

Pls help

curl -X GET "localhost:9200/_cluster/health?pretty"
{
"cluster_name" : "mb_elk",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 885,
"active_shards" : 1770,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}

Welcome to our community! :smiley:

What do your Kibana logs show?

Thanks....

tail -f /data/elasticsearch/elk_logs/mb_elk.log
[2022-06-27T06:16:50,879][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][308911] overhead, spent [263ms] collecting in the last [1s]
[2022-06-27T06:49:20,606][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][310859] overhead, spent [369ms] collecting in the last [1s]
[2022-06-27T07:04:40,548][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][311778] overhead, spent [293ms] collecting in the last [1s]
[2022-06-27T08:10:50,620][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][315743] overhead, spent [287ms] collecting in the last [1s]
[2022-06-27T08:14:59,846][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][315992] overhead, spent [252ms] collecting in the last [1s]
[2022-06-27T09:24:00,784][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][320129] overhead, spent [378ms] collecting in the last [1s]
[2022-06-27T10:39:00,260][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][324623] overhead, spent [332ms] collecting in the last [1.2s]
[2022-06-27T10:41:00,523][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][324743] overhead, spent [536ms] collecting in the last [1.1s]
[2022-06-27T11:30:30,273][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][327709] overhead, spent [333ms] collecting in the last [1.1s]
[2022-06-27T11:57:20,268][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][329317] overhead, spent [252ms] collecting in the last [1s]

System has 16GB memory.

Please share more of the logs. Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you :slight_smile:

Nothing else comming in the log apart from this.

tail -500f /data/elasticsearch/elk_logs/mb_elk.log
[2022-06-27T00:02:20,980][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][286463] overhead, spent [331ms] collecting in the last [1s]
[2022-06-27T00:46:40,415][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][289119] overhead, spent [401ms] collecting in the last [1s]
[2022-06-27T01:07:30,920][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][290368] overhead, spent [400ms] collecting in the last [1s]
[2022-06-27T01:21:10,829][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][291187] overhead, spent [305ms] collecting in the last [1s]
[2022-06-27T01:36:50,815][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][292126] overhead, spent [259ms] collecting in the last [1s]
[2022-06-27T02:09:30,763][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][294084] overhead, spent [338ms] collecting in the last [1s]
[2022-06-27T02:32:10,838][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][295443] overhead, spent [301ms] collecting in the last [1s]
[2022-06-27T02:56:00,335][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][296871] overhead, spent [275ms] collecting in the last [1s]
[2022-06-27T03:07:00,711][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][297531] overhead, spent [260ms] collecting in the last [1s]
[2022-06-27T03:34:30,424][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][299179] overhead, spent [264ms] collecting in the last [1s]
[2022-06-27T03:52:00,423][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][300228] overhead, spent [259ms] collecting in the last [1s]
[2022-06-27T03:56:30,581][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][300498] overhead, spent [263ms] collecting in the last [1s]
[2022-06-27T04:08:20,355][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][301207] overhead, spent [302ms] collecting in the last [1s]
[2022-06-27T04:27:10,942][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][302337] overhead, spent [266ms] collecting in the last [1s]
[2022-06-27T04:29:10,319][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][302456] overhead, spent [265ms] collecting in the last [1s]
[2022-06-27T05:30:01,387][INFO ][o.e.c.m.MetaDataCreateIndexService] [mb_elk.3222] [.monitoring-es-7-2022.06.27] creating index, cause [auto(bulk api)], templates [.monitoring-es], shards [1]/[0], mappings [_doc]
[2022-06-27T05:30:01,391][INFO ][o.e.c.r.a.AllocationService] [mb_elk.3222] updating number_of_replicas to [1] for indices [.monitoring-es-7-2022.06.27]
[2022-06-27T05:30:04,864][INFO ][o.e.c.r.a.AllocationService] [mb_elk.3222] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.monitoring-es-7-2022.06.27][0]] ...]).
[2022-06-27T05:30:05,939][INFO ][o.e.c.m.MetaDataCreateIndexService] [mb_elk.3222] [.monitoring-kibana-7-2022.06.27] creating index, cause [auto(bulk api)], templates [.monitoring-kibana], shards [1]/[0], mappings [_doc]
[2022-06-27T05:30:05,942][INFO ][o.e.c.r.a.AllocationService] [mb_elk.3222] updating number_of_replicas to [1] for indices [.monitoring-kibana-7-2022.06.27]
[2022-06-27T05:30:07,196][INFO ][o.e.c.r.a.AllocationService] [mb_elk.3222] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.monitoring-kibana-7-2022.06.27][0]] ...]).
[2022-06-27T05:43:10,860][INFO ][o.e.m.j.JvmGcMonitorService] [mb_elk.3222] [gc][306893] overhead, spent [261ms] collecting in the last [1s]

Again, please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you :slight_smile:

What is the output from the _cluster/stats?pretty&human API?

{
"cluster_name" : "mb_elk",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 891,
"active_shards" : 1782,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}