Kibana missing primary shards

I have 2 Kibana nodes and the monitoring page keeps saying missing primary shard

GET _cluster/allocation/explain shows nothing to explain

What's the output from _cat/indices?v?

All green

Please don't post pictures of text, they are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them :slight_smile:

Sorry, this is the text output:

health status index                                            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .ds-logs-logstash-algo-2021.05.04-000002         y1SSbiqHTDai_z0MnlC-vw   1   1  140569892            0     50.2gb         25.1gb
green  open   .kibana-event-log-7.12.0-000001                  MSPmC10cQa6oENXBsxQllw   1   1     818155            0    100.3mb           50mb
green  open   .kibana_7.12.0_001                               Z8WuwO2nRdaCh4aoPnrjzg   1   1        339        40122     21.2mb          9.8mb
green  open   .kibana-event-log-7.12.0-000002                  ZLXZ_9txTdmwnd2RYeO_nQ   1   1      44199            0      6.3mb          3.3mb
green  open   .monitoring-kibana-7-mb-2021.05.05               L7fJP84mS_2FJZUZAXMswQ   1   1       2574            0      1.4mb        767.1kb
green  open   .monitoring-es-7-mb-2021.05.03                   vAFCgjFfQLmWMhd4D5JVrA   1   1     481085            0    839.1mb        418.6mb
green  open   .monitoring-kibana-7-mb-2021.05.04               7tSD64ouTXe86qohCD0v2A   1   1      34559            0     17.3mb          8.6mb
green  open   .monitoring-es-7-mb-2021.05.04                   e5bB5IkVS7Gts0JQWvjCWw   1   1     482844            0    848.6mb        424.2mb
green  open   .monitoring-kibana-7-mb-2021.05.07               zvmvjCZfRDmICgltEih8EA   1   1      31742            0     15.7mb          7.8mb
green  open   .kibana_task_manager_7.12.0_001                  TUvzwhD6TRWWkRaPkmhtCA   1   1         24        66246     36.9mb         14.9mb
green  open   .monitoring-es-7-mb-2021.05.05                   Waw9Di0zThmkbe8asAaKmA   1   1      37945            0     67.2mb         33.6mb
green  open   .monitoring-kibana-7-mb-2021.05.06               s1NFGfAiQ9GC53oR7zGbbg   1   1      31753            0     15.6mb          7.8mb
green  open   .monitoring-kibana-7-mb-2021.05.09               G3mDDvMHQA-RBBcl6jYSfw   1   1      34568            0     16.8mb          8.4mb
green  open   .apm-custom-link                                 iGMQTpAXQxyhvavGSENobw   1   1          0            0       416b           208b
green  open   .ds-logs-logstash-fix-messages-2021.05.08-000002 l9StwFHASmWi6T3xfeShTw   1   1          0            0       416b           208b
green  open   .monitoring-kibana-7-mb-2021.05.08               fBDBRaGdQtmV45T-l4i18g   1   1      34560            0     16.8mb          8.4mb
green  open   .ds-logs-logstash-algo-2021.05.07-000003         crzVZGJORDCaGZUYHpvOig   1   1   18856299            0        8gb            4gb
green  open   .monitoring-es-7-mb-2021.05.06                   frpQvIVKQlOmJpzGDerZLw   1   1     445779            0    762.9mb          382mb
green  open   .monitoring-es-7-mb-2021.05.07                   Ohyn5X3dRpubhDIb-x9LrQ   1   1     446112            0    745.2mb        371.2mb
green  open   .monitoring-kibana-7-mb-2021.05.03               WmuVcGgXTfe8YTO0JNO2LQ   1   1      34560            0     17.3mb          8.6mb
green  open   .monitoring-es-7-mb-2021.05.08                   Gq5CtlsKS9-aPshfggSKXw   1   1     553585            0    896.8mb        452.1mb
green  open   .monitoring-es-7-mb-2021.05.09                   xoXGjYxKRhWn7d-9uho_pw   1   1     566086            0    925.5mb        464.4mb
green  open   .apm-agent-configuration                         IVUyMnRGQ86x0WpVwG8fCA   1   1          0            0       416b           208b
green  open   .monitoring-beats-7-2021.05.10                   TLEI_ILsSbmpHnr2Mo8LxA   1   1       7605            0     10.7mb          5.4mb
green  open   .ds-logs-logstash-fix-event-2021.04.08-000001    WDiZp8rNTmCO2t8NQIPKyg   1   1      14600            0      3.4mb          1.7mb
green  open   .tasks                                           z_i7VWfxTOOyomzLQvK7nw   1   1          6            0     46.1kb           23kb
green  open   .monitoring-es-7-mb-2021.05.10                   YT_58Hl7QEWUH8uXAI-l0A   1   1      10725            0     19.7mb          9.8mb
green  open   .ds-logs-logstash-apps-2021.05.07-000006         L-1ZG6inRRu8E0_XeXMI4w   1   1   68498102            0     24.7gb         12.4gb
green  open   .ds-logs-logstash-undefined-2021.04.08-000001    Mba3e2ScQM2Fu0wHTSelbg   1   1    1890448            0    380.3mb        190.2mb
green  open   metricbeat-7.12.0-2021.04.08-000001              cJTAWUL3SVC0Te2zhSTOIQ   1   1    9792465            0      6.2gb          3.1gb
green  open   .monitoring-beats-7-2021.05.05                   WsFDZxq3T3OGM1g7JwxHdQ   1   1      10474            0     14.9mb          7.4mb
green  open   .monitoring-beats-7-2021.05.06                   NVP_bmskRHOEfOxnb14rhg   1   1      39111            0     56.8mb         28.4mb
green  open   .monitoring-beats-7-2021.05.03                   bDDLwfWmSVqW_vIiGEzQbw   1   1      50400            0     67.6mb         33.8mb
green  open   .monitoring-beats-7-2021.05.04                   9t2TIGxKSTqo0_4OfVjbdw   1   1      50400            0     69.4mb         34.7mb
green  open   .monitoring-beats-7-2021.05.09                   3djBou3LShqVNHsqIE0G5A   1   1      50400            0     68.2mb         34.1mb
green  open   .monitoring-beats-7-2021.05.07                   LHJEqhmZT5OivTCHY-feMA   1   1      46147            0     65.1mb         32.5mb
green  open   .monitoring-beats-7-2021.05.08                   jMGk6yDcQ7ar-dsot9mMmg   1   1      50399            0       69mb         34.5mb
green  open   .ds-logs-logstash-fix-2021.04.08-000001          XGEfv9txTOy6uVer6uV6IQ   1   1   15963174            0        5gb          2.5gb
green  open   .ds-logs-logstash-fix-event-2021.05.08-000002    X3h2QJtTT8yumHy2dvY5UA   1   1          0            0       416b           208b
green  open   .ds-logs-logstash-fix-messages-2021.04.08-000001 K6qvyOr6QxCb_kEG7wEEZg   1   1       3170            0        2mb          1.1mb
green  open   .ds-logs-logstash-undefined-2021.05.08-000002    _6GqrkXoRG-LJA7OToxqgQ   1   1          0            0       416b           208b
green  open   .monitoring-logstash-7-mb-2021.05.10             6Ppq2uztQu6mMR6h0wBzRA   1   1      16724            0      2.5mb          1.2mb
green  open   metricbeat-7.12.0-2021.05.08-000002              fZWCRsx5S2OHtnl6WyOhnQ   1   1     651440            0    522.4mb        236.3mb
green  open   .ds-logs-logstash-apps-2021.05.06-000005         62jzeXuNSXiQrWhDQqvnmQ   1   1  132321044            0     50.1gb         24.9gb
green  open   .monitoring-logstash-7-mb-2021.05.04             U_kshaCzS9mOa3inSPZMKQ   1   1     884931            0    100.6mb         50.3mb
green  open   .security-7                                      qpy8Yv0ZTCq07rQ_Tzcb7A   1   1         72            0    426.1kb        202.1kb
green  open   .monitoring-logstash-7-mb-2021.05.03             dd76qfXZRbmZ_aBVSNhYWg   1   1     791133            0     95.7mb         47.7mb
green  open   .monitoring-logstash-7-mb-2021.05.06             7OQNSpKwT-2_uiylS_S4UA   1   1     854161            0     92.6mb         46.3mb
green  open   .monitoring-logstash-7-mb-2021.05.05             XLtWPZAoR2GvVK7ZF9f17w   1   1      66286            0      8.1mb            4mb
green  open   .monitoring-logstash-7-mb-2021.05.08             yH1YqfdlT6SbunGYof2CrA   1   1     950349            0     99.7mb         49.8mb
green  open   .monitoring-logstash-7-mb-2021.05.07             aDE9LHLlTvOF9qDTGT7pmg   1   1     868861            0     97.4mb         48.4mb
green  open   .monitoring-logstash-7-mb-2021.05.09             OglTppikR6i5QeZ2QNuMZA   1   1     950459            0      100mb         49.3mb
green  open   .ds-logs-logstash-fix-2021.05.08-000002          ek3T9jzhRQe9yf-opQRNlw   1   1      64886            0     18.5mb         10.4mb
green  open   .monitoring-kibana-7-mb-2021.05.10               w-qXEjdJRv2g3Tc3L3EZDw   1   1        608            0    665.1kb        339.5kb
green  open   .async-search                                    1wo2VEmnSFSyqinieJTlAg   1   1          0            1      6.7kb          3.3kb
green  open   .monitoring-logstash-7-mb-2021.05.02             2TCqnPaWRcqLmjuhIOehcQ   1   1       3518            0        1mb        510.1kb
1 Like

Thank you!

That's pretty odd, but I have seen Monitoring take some time to age out when things like this change. Is it possible that there were any unassigned shards in the recent history?
Is there anything in your Elasticsearch logs?

Weird, I restarted Kibana and it's green now

image

1 Like

Very weird, but it's sorted :slight_smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.