How to recreate shard failure

Elastic Stack Elasticsearch
1

I am trying to find a way for recreating the shard failure in Elasticsearch
When querying ES can return a 200 response , even if one of the shards had failed to respond.
how do I recreate error, for testing.


{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 1,
    "failures" : [
      {
        "shard" : 1,
        "index" : "hidden_2022-07-19_07_47_32.907435",
        "node" : "J63Vl4b4SC-YrxTFENgijA",
        "reason" : {
          "type" : "node_not_connected_exception",
          "reason" : "[elasticsearch3][172.19.0.2:9300] Node not connected"
        }
      }
    ]
  },
  "hits" : {
    "total" : {
      "value" : 26,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}
2

HI, thanks for asking. For now, I have removed the elastic-stack-alerting label as I don't follow how it relates. I could guess you are doing this testing in the context of some alerts usage? It may help later, but the core question seems unrelated so I'm not deep enough into Elasticsearch to be able to help.

We can ask another team for help on this if you can offer more content - and someone will write back either way, I expect. Let us know. Cheers.

3

Hello, I'm on the Elasticsearch team and may be able to help.

To check, are you trying to simulate the situation in automated tests where only a few shards fail (so you get a partial response)? One option is to create two indices containing some field "field", one with the "field" mapped as "integer", and the other as "keyword". Then you could search across both indices using a "range" query on some non-numeric value -- it will fail on the index shards where the field is mapped as "integer" with a parsing exception.

Here is the general idea:

PUT index
{
  "mappings": {
    "properties": {
      "field": { "type": "integer" }
    }
  }
}

PUT index2
{
  "mappings": {
    "properties": {
      "field": { "type": "keyword" }
    }
  }
}

POST index,index2/_search
{
  "query": {
    "range": {
      "field": {
        "gt": "julie"
      }
    }
  }
}

For our internal testing, we developed a module with a special "error-query" to simulate shard failures and warnings (Add a test module to simulate errors and warnings in search requests by jimczi · Pull Request #71674 · elastic/elasticsearch · GitHub). This is a test module and is only exposed in the daily Elasticsearch snapshot builds.

4

Thanks for taking time to answer.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.