View metrics shows no data for selected host

norgro2601 · February 6, 2020, 1:31pm

Hi,
I struggle with the "View Metrics" display, which shows no data for a limited user, but for the elastic superuser. The user can see the inventory of the hosts, when opening the Metrics app, he can change to Logs and get results, also changing to Uptime works fine. But the "View Metrics" action only gives the header with the host information, but no further metrics. There are data available, as the elastic user gets metrics for the same hosts. I've checked both users out of the same space. I've also checked with Discover, and the results for both users are the same.

Is there a role that needs to be added to the user to get results there ? I'm on 7.5.2 running in Elastic Cloud.

Thanks in advance for any advice.

Best regards,
Norbert

Kerry · February 7, 2020, 12:56pm

Hi @norgro2601,

Sorry to hear you're having problems with the Metrics app.

First, please could you clarify what you mean by "Limited user" - if you could post the role / user / privilege combination you're using for this user that would help us.

Thanks!

norgro2601 · February 7, 2020, 1:36pm

Hi Kerry,

here it is, hope this helps.

Regards,
Norbert

Bild 1

Kerry · February 10, 2020, 10:13am

Thank you.

I think what's happening here is your role isn't providing the privileges to allow access to TSVB. The metrics page you're having problems with issues TSVB requests behind the scenes.

Your screenshot shows the Elasticsearch privileges, but not the Kibana privileges, so I can't be sure you have the correct Kibana privileges. There's some relevant documentation here on assigning feature privileges - the feature in question here would be "Visualize".

norgro2601 · February 10, 2020, 10:51am

I've added Screenshots from Kibana privileges and what I get on the inventory panel and then in the details panel, nothing displayed, although there are data to display.

simianhacker · February 10, 2020, 3:06pm

Can you check to see if Metrics Explorer is working? Use system.cpu.user.pct as the metric and host.name as the grouping field. Can you post a JSON sample of an event with system.cpu.user.pct? Just expand on of the rows from Discover and then click the JSON tab. And finally, can you post the mappings from one of the indices?

norgro2601 · February 10, 2020, 3:53pm

In the metrics expolrer I get a dashlet for each host with "Chart data missing" and "Unable to render chart" in it. Strange thing is, when I open one of the charts in Visualize, I can see the data. I will add the JSON and Mapping in a separate reply

norgro2601 · February 10, 2020, 3:53pm

JSON:

{
"_index": "metricbeat-vw-2020.02.07-000023",
"_type": "_doc",
"_id": "8q-ML3AB193DzneVcvKd",
"_version": 1,
"_score": null,
"_source": {
"@timestamp": "2020-02-10T15:40:37.647Z",
"fields": {
"customer": "VW",
"server": "limeprod01.volkswagenstiftung.de",
"ip": "83.246.48.93"
},
"host": {
"name": "VW Stiftung Survey",
"os": {
"family": "redhat",
"name": "CentOS Linux",
"kernel": "3.10.0-957.5.1.el7.x86_64",
"codename": "Core",
"platform": "centos",
"version": "7 (Core)"
},
"containerized": false,
"ip": [
"192.168.100.20",
"fe80::250:56ff:fe01:b49"
],
"geo": {
"region_iso_code": "DE-NI",
"city_name": "Hannover",
"name": "Volkswagen Stiftung",
"location": "52.3325984,9.7678178",
"continent_name": "Europe",
"country_iso_code": "DE",
"region_name": "Niedersachsen"
},
"architecture": "x86_64",
"id": "9ebe8602e45946819a3fe5754aca0c8f",
"mac": [
"00:50:56:01:0b:49"
],
"hostname": "limeprod01.volkswagenstiftung.de"
},
"service": {
"type": "system"
},
"system": {
"cpu": {
"system": {
"pct": 0.01
},
"steal": {
"pct": 0
},
"total": {
"pct": 0.025
},
"idle": {
"pct": 1.974
},
"iowait": {
"pct": 0.001
},
"cores": 2,
"user": {
"pct": 0.015
},
"nice": {
"pct": 0
},
"softirq": {
"pct": 0
},
"irq": {
"pct": 0
}
}
},
"event": {
"module": "system",
"duration": 355301,
"dataset": "system.cpu"
},
"metricset": {
"name": "cpu",
"period": 10000
},
"tags": [
"VW Stiftung"
],
"ecs": {
"version": "1.1.0"
},
"agent": {
"type": "metricbeat",
"ephemeral_id": "582c6f47-553a-48d3-ae1c-38dc4dd3e576",
"hostname": "limeprod01.volkswagenstiftung.de",
"id": "216d6278-be95-4e18-8011-cef62acae8af",
"version": "7.4.1",
"name": "VW Stiftung Survey"
}
},
"fields": {
"@timestamp": [
"2020-02-10T15:40:37.647Z"
]
},
"sort": [
1581349237647
]
}

norgro2601 · February 10, 2020, 4:02pm

Mapping cannot be posted completely, because it is too long, you'll find "system.cpu" here, is that OK?

      "cpu": {
        "properties": {
          "cores": {
            "type": "long"
          },
          "idle": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              },
              "ticks": {
                "type": "long"
              }
            }
          },
          "iowait": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              },
              "ticks": {
                "type": "long"
              }
            }
          },
          "irq": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              },
              "ticks": {
                "type": "long"
              }
            }
          },
          "nice": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              },
              "ticks": {
                "type": "long"
              }
            }
          },
          "softirq": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              },
              "ticks": {
                "type": "long"
              }
            }
          },
          "steal": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              },
              "ticks": {
                "type": "long"
              }
            }
          },
          "system": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              },
              "ticks": {
                "type": "long"
              }
            }
          },
          "total": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              }
            }
          },
          "user": {
            "properties": {
              "norm": {
                "properties": {
                  "pct": {
                    "type": "scaled_float",
                    "scaling_factor": 1000
                  }
                }
              },
              "pct": {
                "type": "scaled_float",
                "scaling_factor": 1000
              },
              "ticks": {
                "type": "long"
              }
            }
          }
        }
      },

simianhacker · February 10, 2020, 4:41pm

Can you just confirm that host.name and event.dataset are keyword fields.

norgro2601 · February 10, 2020, 6:03pm

Yes, both are keyword fields

simianhacker · February 10, 2020, 6:07pm

I will try and recreate this issue locally to see if I can provide more insight.

simianhacker · February 11, 2020, 3:42pm

I setup a cloud account and created a user that is almost identical to the user you provided and everything seems to be working. Can you look at the Elasticsearch > Logs page in your cloud account and see if there are any errors?

norgro2601 · February 11, 2020, 4:14pm

No errors, just 2 warnings about deprecation came up, when I tried to view the metrics, but I'm not sure, if they are related to the function, because the instance 82 is a warm node.

norgro2601 · February 11, 2020, 4:29pm

I've now tried another setup, I've added the kibana_user role to the user, and immediately it works. So there must be something necessary missing in the vw_user role.

norgro2601 · February 11, 2020, 4:33pm

Here is the role setup from the API:

{
"VW_user" : {
"cluster" : ,
"indices" : [
{
"names" : [
"filebeat-vw-",
"metricbeat-vw-",
"logstash-vw-",
"heartbeat-vw-"
],
"privileges" : [
"read"
],
"field_security" : {
"grant" : [
"*"
],
"except" :
},
"allow_restricted_indices" : false
}
],
"applications" : [
{
"application" : "kibana-.kibana",
"privileges" : [
"feature_discover.read",
"feature_dashboard.read",
"feature_infrastructure.read",
"feature_logs.read",
"feature_uptime.read",
"feature_siem.read",
"feature_visualize.read"
],
"resources" : [
"space:volkswagen-stiftung"
]
}
],
"run_as" : ,
"metadata" : { },
"transient_metadata" : {
"enabled" : true
}
}
}

simianhacker · February 11, 2020, 5:15pm

Good News! I've recreated the issue on my cloud account. I'm going to dig into this and see if I can come up with a work around for you.

simianhacker · February 11, 2020, 5:27pm

Ok... Here is what I came up with. When I assigned my test user just the VW_user role I could see the Metrics UI but I couldn't see the charts in Metrics Explorer or the charts on the metric detail page. BUT if I also added the kibana_user role to my test user, everything started working as expected.

simianhacker · February 11, 2020, 6:08pm

Sorry... I wasn't happy with just applying the kibana_user role to the test user so I did some more poking around and found this combination works:

{
  "vw_test_role" : {
    "cluster" : [ ],
    "indices" : [
      {
        "names" : [
          "metricbeat-*"
        ],
        "privileges" : [
          "read"
        ],
        "field_security" : {
          "grant" : [
            "*"
          ],
          "except" : [ ]
        },
        "allow_restricted_indices" : false
      }
    ],
    "applications" : [
      {
        "application" : "kibana-.kibana",
        "privileges" : [
          "feature_infrastructure.read"
        ],
        "resources" : [
          "space:vw-test-space"
        ]
      },
      {
        "application" : "kibana-.kibana",
        "privileges" : [
          "feature_infrastructure.read"
        ],
        "resources" : [
          "space:default"
        ]
      }
    ],
    "run_as" : [ ],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}

I'm gonna do some more digging on our end to see why the default space permission need to be there.

simianhacker · February 11, 2020, 8:31pm

@norgro2601 I've tracked down the root cause, there is an issue on our end where the call to TSVB's backend is being made under the Default space. We are looking into addressing this issue. For now, the work around is to add the Default space to your role with the same restrictions as your custom space.

Sorry for the inconvenience.

Topic		Replies	Views
Visualizations not updating for some hosts Kibana	3	365	September 15, 2020
Kibana user not able to see the dashboard data Kibana	6	2595	April 28, 2020
Host Overview Dashboard not showing data for a host Kibana	3	797	June 6, 2018
Metricbeat Module is not visible in Infrastructure Host Overview Metrics	5	669	August 31, 2020
I am unable to see system metrics of more than one host Beats metricbeat	3	508	October 15, 2020

View metrics shows no data for selected host

Related Topics