# Stack monitoring cannot see Indices

**URL:** <https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-monitoring\
**Created:** [January 9, 2020, 6:41pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478 "2020-01-09T18:41:21Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 9, 2020, 6:41pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/1 "2020-01-09T18:41:21Z")

</div>

Hi,

I am ES 7.5 and using metricbeat for monitoring, under my cluster, Elasticsearch, Indices. I can see the number of Indices, Memory etc but in the filter list I see nothing, I have changed the date range to days and still nothing, if I flip to view System indices I see those.

Any ideas?

Thanks  
Phil

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 10, 2020, 6:31am UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/2 "2020-01-10T06:31:02Z")

</div>

Hi @probson,

Are you using the `elasticsearch` or `elasticsearch-xpack` Metricbeat module? Either way, can you share that config (found in the `modules.d` folder, like`modules.d/elasticsearch-xpack.yml`)

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 10, 2020, 9:41am UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/3 "2020-01-10T09:41:04Z")

</div>

I am just using the default settings

- module: elasticsearch  
metricsets:
  - ccr
  - cluster\_stats
  - enrich
  - index
  - index\_recovery
  - index\_summary
  - ml\_job
  - node\_stats
  - shard  
period: 10s

Thanks  
Phil

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 10, 2020, 2:52pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/4 "2020-01-10T14:52:58Z")

</div>

Hmm

Let's see what the monitoring data shows.

Can you run the following query against the monitoring cluster and return the results?

```auto
POST .monitoring-es-*/_search
{
  "size": 0, 
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "type": "index_stats"
          }
        }
      ]
    }
  },
  "aggs": {
    "clusters": {
      "terms": {
        "field": "cluster_uuid",
        "size": 20
      },
      "aggs": {
        "indices": {
          "terms": {
            "field": "index_stats.index",
            "size": 500
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 17, 2020, 2:43pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/5 "2020-01-17T14:43:54Z")

</div>

Hi,

Sorry for the delay, below as requested:

{  
"took" : 216,  
"timed\_out" : false,  
"\_shards" : {  
"total" : 7,  
"successful" : 7,  
"skipped" : 0,  
"failed" : 0  
},  
"hits" : {  
"total" : {  
"value" : 10000,  
"relation" : "gte"  
},  
"max\_score" : null,  
"hits" :   
},  
"aggregations" : {  
"clusters" : {  
"doc\_count\_error\_upper\_bound" : 0,  
"sum\_other\_doc\_count" : 0,  
"buckets" : [  
{  
"key" : "D56cfJfWScKYkVmhg9yRPQ",  
"doc\_count" : 1447661,  
"indices" : {  
"doc\_count\_error\_upper\_bound" : 0,  
"sum\_other\_doc\_count" : 0,  
"buckets" : [  
{  
"key" : ".monitoring-alerts-7",  
"doc\_count" : 57131  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.11",  
"doc\_count" : 57130  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.11",  
"doc\_count" : 57130  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.11",  
"doc\_count" : 57130  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.10",  
"doc\_count" : 52200  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.10",  
"doc\_count" : 52200  
},  
{  
"key" : ".monitoring-logstash-7-2020.01.10",  
"doc\_count" : 52200  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.10",  
"doc\_count" : 52200  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.12",  
"doc\_count" : 48490  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.12",  
"doc\_count" : 48490  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.12",  
"doc\_count" : 48490  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.09",  
"doc\_count" : 43560  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.09",  
"doc\_count" : 43560  
},  
{  
"key" : ".monitoring-logstash-7-2020.01.09",  
"doc\_count" : 43560  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.09",  
"doc\_count" : 43560  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.13",  
"doc\_count" : 39850  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.13",  
"doc\_count" : 39850  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.13",  
"doc\_count" : 39850  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.08",  
"doc\_count" : 34920  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.08",  
"doc\_count" : 34920  
},  
{  
"key" : ".monitoring-logstash-7-2020.01.08",  
"doc\_count" : 34920  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.08",  
"doc\_count" : 34920  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.14",  
"doc\_count" : 31210  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.14",  
"doc\_count" : 31210  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.14",  
"doc\_count" : 31210  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.07",  
"doc\_count" : 26280  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.07",  
"doc\_count" : 26280  
},  
{  
"key" : ".monitoring-logstash-7-2020.01.06",  
"doc\_count" : 26280  
},  
{  
"key" : ".monitoring-logstash-7-2020.01.07",  
"doc\_count" : 26280  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.07",  
"doc\_count" : 26280  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.15",  
"doc\_count" : 22570  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.15",  
"doc\_count" : 22570  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.15",  
"doc\_count" : 22570  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.06",  
"doc\_count" : 17640  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.06",  
"doc\_count" : 17640  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.06",  
"doc\_count" : 17640  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.16",  
"doc\_count" : 13930  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.16",  
"doc\_count" : 13930  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.16",  
"doc\_count" : 13930  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.05",  
"doc\_count" : 9000  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.05",  
"doc\_count" : 9000  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.05",  
"doc\_count" : 9000  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.17",  
"doc\_count" : 5290  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.17",  
"doc\_count" : 5290  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.17",  
"doc\_count" : 5290  
},  
{  
"key" : ".monitoring-es-7-mb-2020.01.04",  
"doc\_count" : 360  
},  
{  
"key" : ".monitoring-kibana-7-mb-2020.01.04",  
"doc\_count" : 360  
},  
{  
"key" : ".monitoring-logstash-7-mb-2020.01.04",  
"doc\_count" : 360  
}  
]  
}  
}  
]  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 17, 2020, 2:55pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/6 "2020-01-17T14:55:41Z")

</div>

Something i just thought about, x-pack is using an account with the role : 'remote\_monitoring\_agent`

Would have any affect on this?

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 22, 2020, 7:14pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/7 "2020-01-22T19:14:10Z")

</div>

That shouldn't matter.

`D56cfJfWScKYkVmhg9yRPQ` - do you know if that is the cluster you're looking at in the monitoring UI?

Can you possible share a screenshot as well so I can make sure I'm on the right page?

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 23, 2020, 11:24am UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/8 "2020-01-23T11:24:27Z")

</div>

Hi,

Hopefully this is what you needed, cluster name is soc-cluster, there are no other clusters

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/9/19fe627f07de3d24431704b76f30cb4bd2f7e4c9.png)

Thanks  
Phil

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 27, 2020, 5:22pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/9 "2020-01-27T17:22:14Z")

</div>

Are you using a single ES cluster? or do you have multiple?

The results from your query indicate you _only_ have system indices on that ES cluster

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 27, 2020, 5:50pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/10 "2020-01-27T17:50:00Z")

</div>

Hi,

I only have 1 cluster.

Kind regards  
Phil

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 27, 2020, 6:00pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/11 "2020-01-27T18:00:46Z")

</div>

Okay great.

If you run `GET _cat/indices`, what do you see?

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 27, 2020, 9:27pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/12 "2020-01-27T21:27:16Z")

</div>

Hi,

With that i can see all of my indices not just the .monitoring ones.

Thanks  
Phil

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 28, 2020, 6:35pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/13 "2020-01-28T18:35:53Z")

</div>

If you run `GET _stats/docs,fielddata,indexing,merge,search,segments,store,refresh,query_cache,request_cache`, do you see your indices in that list?

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 29, 2020, 11:06am UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/14 "2020-01-29T11:06:57Z")

</div>

> [@chrisronline](#):
>
> GET \_stats/docs,fielddata,indexing,merge,search,segments,store,refresh,query\_cache,request\_cache

Hi, i do indeed.

Kind regards  
Phil

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 29, 2020, 2:48pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/15 "2020-01-29T14:48:18Z")

</div>

Very strange.

I'm assuming you've already checked this, but are there any errors in the metricbeat logs? Maybe share the start up log?

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 29, 2020, 3:21pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/16 "2020-01-29T15:21:55Z")

</div>

Hi Chris,

This morning i added the role:remote\_monitoring\_collector to the account the x-pack monitoring is running as, just checked now (your post reminded me to check) the indices are now appearing. I believe this is an extract from the guide i originally followed:

[https://www.elastic.co/guide/en/elasticsearch/reference/current/esms.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/esms.html)

Create a user on the production cluster that has the remote\_monitoring\_collector built-in role. Alternatively, use the remote\_monitoring\_user built-in user.

Sorry that i did not try this earlier on.

Thanks  
Phil

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 29, 2020, 3:37pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/17 "2020-01-29T15:37:44Z")

</div>

I'm glad you were able to resolve it!

Is this the same account you configured in your metricbeat stack modules? Like `elasticsearch-xpack.yml`?

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 29, 2020, 3:50pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/18 "2020-01-29T15:50:37Z")

</div>

Hi,

It is indeed.

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [January 29, 2020, 4:06pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/19 "2020-01-29T16:06:06Z")

</div>

Hmm interesting. If it were a permission issue, I'd expect to see something in the metricbeat server log file about it, but nothing was there?

---

<div class="post-metadata">

**Author:** ![probson](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@probson](https://discuss.elastic.co/u/probson)\
**Post date:** [January 29, 2020, 4:42pm UTC](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478/20 "2020-01-29T16:42:06Z")

</div>

Hi,

The metricbeat logs have very little in, no errors  
This is the only error i can see in the service log.  
Error fetching data for metricset elasticsearch.enrich: HTTP error 403 in : 403 Forbidden

Thanks  
Phil

[Next page](https://discuss.elastic.co/t/stack-monitoring-cannot-see-indices/214478.md?page=2)