_cat/indices returns hidden indices

Elastic Stack Elasticsearch
1

Hi,

On a fresh setup of version 8.15.0, I run the following query:

GET /_cat/indices?v

It returns a bunch of internal indices:

health status index                                                              uuid                   pri rep docs.count docs.deleted store.size pri.store.size dataset.size
green  open   .internal.alerts-transform.health.alerts-default-000001            U6p45fZLTAW-Yu3f0gH6Ug   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-observability.logs.alerts-default-000001          GcryognTSEuMeNnMifOhWw   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-observability.uptime.alerts-default-000001        BiblXyo1RE-4lSRJ0JyDOQ   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-ml.anomaly-detection.alerts-default-000001        1FMZ68VrSFePQ6LdC3wDkw   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-observability.slo.alerts-default-000001           ORZ9FRz3Q-C7jVONc5WBwg   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-default.alerts-default-000001                     wTWVWlkmSK-V4GWXkNC1_g   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-observability.apm.alerts-default-000001           20nQ5SU8QYmcWm5dJfmfOQ   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-observability.metrics.alerts-default-000001       yrSqoEs1RJiftdgDqbjMRg   1   0          0            0       249b           249b         249b
green  open   .kibana-observability-ai-assistant-conversations-000001            tVXNSoJoTHm49XZbc6j4XQ   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-ml.anomaly-detection-health.alerts-default-000001 PczWPdSeSjGL4BZvggkTfA   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-observability.threshold.alerts-default-000001     T8UiycvPSvi3xLz6E3iVYg   1   0          0            0       249b           249b         249b
green  open   .kibana-observability-ai-assistant-kb-000001                       HX1Vn5grQrCBHZ7f2HvuUw   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-security.alerts-default-000001                    lMwhO-MgTO2q3nRZS2d9PQ   1   0          0            0       249b           249b         249b
green  open   .internal.alerts-stack.alerts-default-000001                       uf310DEsSuCZKFyg4xnBGg   1   0          0            0       249b           249b         249b

Checking the settings for these indices, they are hidden. For example:

GET .internal.alerts-observability.slo.alerts-default-000001

Returns:

{
    ".internal.alerts-observability.slo.alerts-default-000001": {
        "settings": {
            "index": {
                ...
                "hidden": "true",
                ...
            }
        }
    }
}

So despite them being hidden, they are still returned. So I tried:

GET /_cat/indices?v&expand_wildcards=open
GET /_cat/indices?v&expand_wildcards=open,closed

The indices are still part of the results. The documentation states:

expand_wildcards:
open: Match open, non-hidden indices. Also matches any non-hidden data stream.
closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.

So, non-hidden indices should be matched. Why do I still get hidden indices back, then?

If I add hidden as follows, then I get more internal indices back - indices that are otherwise filtered out. So it seems that some/many internal indices are filtered as I would expect, but not all of them.

GET /_cat/indices?v&expand_wildcards=open,closed,hidden

Furthermore, running the below query shows that the indices in question are actually queried with a search query as well, despite being hidden. Regardless of adding the expand_wildcards query parameter as well. So I guess this is not specific to the CAT API.

GET /_search
{
  "profile": true, 
  "query": {
    "match_all": {}
  }
}

This has actually annoyed me for a while. I might be overlooking something, but I don't think the behavior is obvious based on the documentation. Are these indices "special" in some way, or why am I seeing this behavior? Sorry if I missed something obvious.

Thanks!

2

The behavior is a little surprising -- when you ask for non-hidden indices (the default), it returns all non-hidden indices plus any indices backing non-hidden data streams or aliases.
Looking at your example of .internal.alerts-observability.slo.alerts-default-000001, the response of GET .internal.alerts-observability.slo.alerts-default-000001 shows that it has an alias named .alerts-observability.slo.alerts-default. And GET _alias/.alerts-observability.slo.alerts-default does not return "hidden": "true", meaning that the alias is not hidden.

1 Like
3

Thank you for the reply, @Keith_Massey! That's absolutely correct. I just find this surprising, since I literally just started up Elasticsearch and Kibana for the first time and ran the queries immediately. I would just expect GET /_cat/indices to return nothing and for all of this internal stuff to be hidden by default.

Do you know if the aliases are supposed to be hidden? I would guess so, since this is all internal stuff that doesn't seem in line with other system indices/aliases/data streams. Admittedly I haven't dug into the inner workings of this.

4

Bump. Still looking for an answer to this (and I know others are wondering as well) :slightly_smiling_face:

5

Hello,

And if you want to search by indexes, excluding specific internal indexes?

GET /_cat/indices?v&index=*,-.*,-.internal.*,-.*default*