Unassigned shards

Elastic Stack Elasticsearch
1

Hi mates,
I've made a mistake in my ELK deployment. I was trying to install a new elastic agent and I put the wrong token and used the one used to configure a new server.
Now I've got a lot of unassigned shards and I don't know how to fix them. Furthermore, ELK goes very slow now and Kibana is reporting always the same error, no matter the command I write:

GET _cluster/allocation/explain
{"statusCode":502,"error":"Bad Gateway","message":"Client request timeout"}

The only command I get a different response is:

GET _search
{
  "query": {
    "match_all": {}
  }
}

{
  "took" : 42,
  "timed_out" : false,
  "_shards" : {
    "total" : 363,
    "successful" : 341,
    "skipped" : 0,
    "failed" : 22,
    "failures" : [
      {
        "shard" : 0,
        "index" : ".ds-logs-elastic_agent.metricbeat-default-2023.09.19-000032",
        "node" : null,
        "reason" : {
          "type" : "no_shard_available_action_exception",
          "reason" : null,
          "index_uuid" : "_pSoSn8uSGefpM0t1pB8xw",
          "shard" : "0",
          "index" : ".ds-logs-elastic_agent.metricbeat-default-2023.09.19-000032"
        }
      },
      {
        "shard" : 0,
        "index" : ".ds-logs-squid.log-server-2023.08.30-000005",
        "node" : null,
        "reason" : {
          "type" : "no_shard_available_action_exception",
          "reason" : null,
          "index_uuid" : "-g69dXFKTKeBDv8-XcvPmA",
          "shard" : "0",
          "index" : ".ds-logs-squid.log-server-2023.08.30-000005"
        }
      },
      {
        "shard" : 0,
        "index" : ".ds-logs-system.security-default-2023.09.19-000034",
        "node" : null,
        "reason" : {
          "type" : "no_shard_available_action_exception",
          "reason" : null,
          "index_uuid" : "r_vb24oNTEC_9KoHDZ-czw",
          "shard" : "0",
          "index" : ".ds-logs-system.security-default-2023.09.19-000034"
        }
      },
      {
        "shard" : 0,
        "index" : ".ds-logs-system.syslog-fleetserver-2023.09.19-000010",
        "node" : "dnaT5yKwRVSDF9egWtz0qg",
        "reason" : {
          "type" : "no_shard_available_action_exception",
          "reason" : "[vmw3le1es][172.24.0.16:9300][indices:data/read/search[phase/query]]",
          "index_uuid" : "vTNZDjETR0mSowbqVPGrdQ",
          "shard" : "0",
          "index" : ".ds-logs-system.syslog-fleetserver-2023.09.19-000010"
        }
      },
      {
        "shard" : 0,
        "index" : ".ds-metrics-elastic_agent.elastic_agent-default-2023.09.19-000034",
        "node" : null,
        "reason" : {
          "type" : "no_shard_available_action_exception",
          "reason" : null,
          "index_uuid" : "GCYWyVLyRPKVj8fkwr8iww",
          "shard" : "0",
          "index" : ".ds-metrics-elastic_agent.elastic_agent-default-2023.09.19-000034"
        }
      },
      {
        "shard" : 0,
        "index" : ".ds-metrics-elastic_agent.elastic_agent-fleetserver-2023.09.19-000012",
        "node" : "dnaT5yKwRVSDF9egWtz0qg",
        "reason" : {
          "type" : "no_shard_available_action_exception",
          "reason" : "[vmw3le1es][172.24.0.16:9300][indices:data/read/search[phase/query]]",
          "index_uuid" : "jbMPrM61R02ff7nmYI5IRw",
          "shard" : "0",
          "index" : ".ds-metrics-elastic_agent.elastic_agent-fleetserver-2023.09.19-000012"
        }
      },
      {
        "shard" : 0,
        "index" : ".ds-metrics-elastic_agent.elastic_agent-server-2023.09.15-000006",
        "node" : null,
        "reason" : {
          "type" : "no_shard_available_action_exception",
          "reason" : null,
          "index_uuid" : "1IxxzYihTPe6uMrDlgQnYA",
          "shard" : "0",
          "index" : ".ds-metrics-elastic_agent.elastic_agent-server-2023.09.15-000006"
        }
      },

and so on with all the unassigned shards.
At the beginning of this problem, I had a lot of unassigned shards, most of them have been auto fixed but I always have around 25 unassigned.

I really don't mind missing the information as long as I can restore the ELK functionality as the elastic agents will keep reporting and sending updated information.

Thanks a lot in advance for any help!!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.