Getting "stats[n] does not support wildcards" for _stats ES API

We have setup an ES cluster(version 8.11.0) using ECK with xpack.security enabled and are getting the following error while trying to hit the "/_stats" API -

Sample curl request -

curl --location 'http://localhost:9200/_stats'

Error response -

{
    "_shards": {
        "total": 10,
        "successful": 0,
        "failed": 10,
        "failures": [
            {
                "shard": 0,
                "index": "test_index",
                "status": "INTERNAL_SERVER_ERROR",
                "reason": {
                    "type": "failed_node_exception",
                    "reason": "Failed node [5-i-JJzsR6OPmdlr4Pc0ag]",
                    "node_id": "5-i-JJzsR6OPmdlr4Pc0ag",
                    "caused_by": {
                        "type": "illegal_argument_exception",
                        "reason": "the action indices:monitor/stats[n] does not support wildcards; the provided index expression(s) [*] are not allowed"
                    }
                }
            }
        ]
    },
    "_all": {
        "primaries": {},
        "total": {}
    },
    "indices": {}
}

When we try to get a specific index's stats, that works fine -

Sample curl request -

curl --location 'http://localhost:9200/test_index/_stats'

Snippet of the successful response -

{
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "_all": {
        "primaries": {
            "docs": {
                "count": 14097379,
                "deleted": 1151323
            },
            ...
        },
        "total": {
            "docs": {
                "count": 14097379,
                "deleted": 1151323
            },
            ...
        }
    },
    "indices": {
        "test_index": {
            "uuid": "NhyCjP_QQLC8TZE24GpvcQ",
            "health": "green",
            "status": "open",
            "primaries": {
                "docs": {
                    "count": 14097379,
                    "deleted": 1151323
                },
                ...
            },
            "total": {
                "docs": {
                    "count": 14097379,
                    "deleted": 1151323
                },
                ...
            }
        }
    }
}

We have an existing ES cluster(version 7.16.3) with xpack.security disabled and the aforementioned "/_stats" request works fine there.

Can someone from the Elastic team please help confirm if we need to explicitly enable support for wildcards if xpack.security is enabled; and if yes, how can we achieve that since we could not find any supporting documentation.

Perhaps per the docs

If the Elasticsearch security features are enabled, you must have the monitor or manage index privilege for the target data stream, index, or alias.

Perhaps your user does not have those permissions

Thanks, @stephenb .

Yes, the permissions should not be a problem since the following APIs are returning a valid response -

GET /<target>/_stats/<index-metric>

GET /<target>/_stats

The only one where we are seeing the "Internal server error" specifically related to wildcard usage is -

GET /_stats

Hmm, weird.... It works for me as expected, same version both with security enabled and without security enabled.

So on a cluster with no security at all, it works...

hyperion:~ sbrown$ curl  http://localhost:9200/
{
  "name" : "ebcb74a532a0",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "LiJpuuoJR2WiOzyi796HrA",
  "version" : {
    "number" : "8.11.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "d9ec3fa628c7b0ba3d25692e277ba26814820b20",
    "build_date" : "2023-11-04T10:04:57.184859352Z",
    "build_snapshot" : false,
    "lucene_version" : "9.8.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
hyperion:~ sbrown$ curl  http://localhost:9200/_stats?pretty
{
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "docs" : {
        "count" : 8,
        "deleted" : 0
      },
      "shard_stats" : {
        "total_count" : 2
      },
      "store" : {
        "size_in_bytes" : 91290,
        "total_data_set_size_in_bytes" : 91290,
        "reserved_in_bytes" : 0
      },

And on a cluster with full security, it works...

hyperion:~ sbrown$ 
hyperion:~ sbrown$ curl -k -u elastic:mypassword https://localhost:9200
{
  "name" : "es01",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "DCBYiyU1ST-l5tDxM7tquw",
  "version" : {
    "number" : "8.11.0",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "d9ec3fa628c7b0ba3d25692e277ba26814820b20",
    "build_date" : "2023-11-04T10:04:57.184859352Z",
    "build_snapshot" : false,
    "lucene_version" : "9.8.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
hyperion:~ sbrown$ curl -k -u elastic:mypassword https://localhost:9200/_stats?pretty
{
  "_shards" : {
    "total" : 11,
    "successful" : 11,
    "failed" : 0
  },
  "_all" : {
    "primaries" : {
      "docs" : {
        "count" : 0,
        "deleted" : 0
      },
      "shard_stats" : {
        "total_count" : 11
      },
      "store" : {
        "size_in_bytes" : 2497,
        "total_data_set_size_in_bytes" : 2497,
        "reserved_in_bytes" : 0
      },
      "indexing" : {
        "index_total" : 0,
        "index_time_in_millis" : 0,
        "index_current" : 0,
......

I noticed when you have security enabled, you were not passing a username in the curl request... not sure if that was intentional ... or how you are authenticating.

We are using the readonlyrest plugin with our cluster, could that be causing this erroneous behavior? Since that is used only for authentication, I presumed it should not be throwing the wildcard usage error.

I noticed when you have security enabled, you were not passing a username in the curl request... not sure if that was intentional ... or how you are authenticating.

Yes, even without providing the credentials, the /_stats specific API calls work and there is no access issue and we get the response for /<target>/_stats.

I did try by explicitly passing the readonlyrest user, but it yielded the same response.

I do not use / not experienced with ROR ... but if that is the different between yours and mine that would be my first suspect :slight_smile:

@shweta.c commenting about your post from 2022: ES 7.17 | Exponentially growing query cache

The issue was fixed in this PR Make LRUQueryCache respect Accountable queries on eviction and consisten… by gtroitskiy · Pull Request #12614 · apache/lucene · GitHub

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.