# General Log Warning Question
**URL:** https://discuss.elastic.co/t/general-log-warning-question/338443
**Category:** Beats
**Tags:** metricbeat
**Created:** [July 14, 2023, 9:04pm UTC](https://discuss.elastic.co/t/general-log-warning-question/338443 "2023-07-14T21:04:50Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Ray3](https://discuss.elastic.co/u/Ray3)
#### Post date: [July 14, 2023, 9:04pm UTC](https://discuss.elastic.co/t/general-log-warning-question/338443/1 "2023-07-14T21:04:50Z")
I deployed metricbeat to a node. Unless I use superuser role, I will get warnings in the log, that it cannot take certain actions as the api key used is unauthorized. I do see data reported in kibana, it appears metrics are getting logged.
The question is, are some warning normal? Or should I keep adding privilages until all I have are info messages?
Thanks
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [July 15, 2023, 5:22am UTC](https://discuss.elastic.co/t/general-log-warning-question/338443/2 "2023-07-15T05:22:51Z")
Hi @Ray3 welcome to the community!
We need a bot more detail to help.
What Version?
How did you install?
How are you starting metricbeat?
Could you share your metricbeat.yml and any other modules you have enabled?
And we need to see the actual log message
---
### Author:  [@Ray3](https://discuss.elastic.co/u/Ray3)
#### Post date: [July 17, 2023, 5:39pm UTC](https://discuss.elastic.co/t/general-log-warning-question/338443/3 "2023-07-17T17:39:26Z")
Thanks for reaching out - I think I am making progress. I am running 8.8.1 elasticsearch and beats agents. Installed using APT. sysctrl starts metricbeat service on a cluster node.
I am starting metricbeat using api value from key store.
Mostly trying to learn how to fish - Does the /modules.d/elasticsearch-xpack.yml override the logon credentials stored in the metricbeat.yml?
Is the module loaded into a separate running service? If not, I was thinking about combining to a single api key and use the same credential in the api\_key for both yaml files.
Using the template from the elastic doc, built the following:
```auto
POST /_security/api_key
{
"name": "Cluster_Node_Beat_Agents",
"expiration": "3600d",
"role_descriptors": {
"Metricbeat_Publish": {
"cluster": [
"monitor",
"read_ilm",
"read_pipeline"
],
"index": [
{
"names": [
"metricbeat-*"
],
"privileges": [
"view_index_metadata",
"create_doc"
]
}
]
},
"Node_Based_Cluster_Monitoring": {
"cluster": [
"manage_index_templates",
"manage_ingest_pipelines",
"monitor",
"cluster:admin/ilm/get",
"cluster:admin/ilm/put",
"cluster:monitor/xpack/watcher/watch/get",
"cluster:admin/xpack/watcher/watch/put",
"cluster:admin/xpack/watcher/watch/delete"
],
"indices": [
{
"names": [
".monitoring-*"
],
"privileges": [
"all"
],
"allow_restricted_indices": false
},
{
"names": [
"metricbeat-*"
],
"privileges": [
"index",
"create_index",
"view_index_metadata",
"indices:admin/aliases",
"indices:admin/rollover"
],
"allow_restricted_indices": false
}
]
}
}
}
```
However, I then get the error:
Jul 17 10:35:00 dtcepvp-elkna metricbeat[25159]: {"log.level":"error","@timestamp":"2023-07-17T10:35:00.211-0700","log.origin":{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset elasticsearch.index: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized","service.name":"metricbeat","ecs.version":"1.6.0"}
So, it seems the remote\_monitoring\_agent role has something more that I cannot add to my api\_key to get it to work.
I think I am just being stubborn, I would prefer to use an api key, but maybe should just use username and password for agent logons...
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [July 17, 2023, 6:09pm UTC](https://discuss.elastic.co/t/general-log-warning-question/338443/4 "2023-07-17T18:09:01Z")
> [@Ray3](#):
>
> Mostly trying to learn how to fish - Does the /modules.d/elasticsearch-xpack.yml override the logon credentials stored in the metricbeat.yml?
>
> Is the module loaded into a separate running service? If not, I was thinking about combining to a single api key and use the same credential in the api\_key for both yaml files.
Hi @Ray3 Think perhaps there is some common confusion:
1. metricbeat needs and elasticsearch cluster to talk to, that is what is in the output section of the metrcbeat.yml
2. If you are monitoring and elasticsearch cluster which may or MAY NOT be (which is quite often the case) the same cluster you are sending the metrics to you need credentials / API key for that.
3. Yes you can put the module directly into the metricbeat.yml if you please. No modules are not a separate process space. (could be multiple threads though)
Hope this helps
---
### Author:  [@Ray3](https://discuss.elastic.co/u/Ray3)
#### Post date: [July 17, 2023, 6:56pm UTC](https://discuss.elastic.co/t/general-log-warning-question/338443/5 "2023-07-17T18:56:11Z")
Thanks again Stephen,
I have tested creating a user account with proper roles - and all is well if I configure both yml files to logon by username and password. however, as soon as I create an API key, cluster monitoring begins to show errors.
So taking a working username and password, tried to create an api key for logon for those credentials such as in the example:
```auto
POST /_security/api_key/grant
{
"grant_type": "password",
"username": "Cluster_Node_Beats_Agents",
"password": "testtest",
"api_key" : {
"name": "Cluster_Node_Beats_Agents"
}
}
```
But then the errors return. It looks like for cluster monitoring, (remote\_monitoring\_agent role) the modules yml file, is not happy with an api\_key, no matter how I go about it.
I get the sense most are just using username and password and then protecting those values in the key store when monitoring a cluster with a beats agent?
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [July 17, 2023, 8:15pm UTC](https://discuss.elastic.co/t/general-log-warning-question/338443/6 "2023-07-17T20:15:07Z")
I think it is a limitation...
I tried with a super user api\_key and could not get it to work.... asking internally... agree seems odd.
EDIT Yup limitation that looks like it is going to be addressed in an upcoming release you can track it here
>
>
> \### Summary
> The metricbeat \[elasticsearch\](https://www.elastic.co/guide/en/beat…s/metricbeat/8.5/securing-communication-elasticsearch.html) module allows one to specify api\_key configuration but it is only working for elasticsearch output and not the elasticsearch monitoring module.
>
> Since the \[elastic integration\](https://github.com/elastic/integrations/tree/main/packages/elasticsearch) spawns the metricbeat module under the hood, one should be able to set any accepted metricbeat parameters from the integration.
>
> \### Implementation hints
> \- The parameters are defined under the integration's manifest \[vars block\](https://github.com/elastic/integrations/blob/main/packages/elasticsearch/manifest.yml#L27)
> \- The parameters are then transformed into a metricbeat configuration in the data streams \[stream.yml.hbs file\](https://github.com/elastic/integrations/blob/main/packages/elasticsearch/data\_stream/ccr/agent/stream/stream.yml.hbs)
>
> \> Did we verify that api keys where accepted by metricbeat module ? This opened issue says otherwise \[elastic/beats#29271\](https://github.com/elastic/beats/issues/29271)
>
> After taking a look at metricbeat module it seems that it doesn't support api\_key authorization which is also proven by the ticket Kevin linked above however it seems to be working for the output module.
>
> The necessary changes as per my understanding are as follows:
>
> 1. Add APIKey string prop \[here\](https://github.com/elastic/beats/blob/main/metricbeat/mb/mb.go#L275) which will hold the raw decoded api\_key value.
> 2. Read the config api\_key \[here\](https://github.com/elastic/beats/blob/main/metricbeat/mb/parse/url.go#L74) similar to username and pass.
> 3. checking \[here\](https://github.com/elastic/beats/blob/main/metricbeat/mb/parse/url.go#L94) if both Username, Password & api\_key are both set then reject similar to \[this\](https://github.com/elastic/beats/blob/main/libbeat/esleg/eslegclient/config.go#L60).
> 4. Keep \[cascading\](https://github.com/elastic/beats/blob/main/metricbeat/mb/parse/url.go#L109) the api\_key prop through the different functions similar to username and password so that you are able to access it \[here\](https://github.com/elastic/beats/blob/main/metricbeat/helper/http.go#L79).
> 5. Add Auth header with the encoded api\_key value, similar to what is done \[here\](https://github.com/elastic/beats/blob/main/libbeat/esleg/eslegclient/connection.go#L165) and \[here\](https://github.com/elastic/beats/blob/main/libbeat/esleg/eslegclient/connection.go#L428).
> 6. We need to verify what should happen if both \[BearerTokenFile\](https://github.com/elastic/beats/blob/main/metricbeat/helper/http.go#L72) & APIKey are set, should we just overwrite the value with what we have in the api\_key prop or reject similar to step 3.
> 7. Update affected tests
---
### Author:  [@Ray3](https://discuss.elastic.co/u/Ray3)
#### Post date: [July 17, 2023, 8:38pm UTC](https://discuss.elastic.co/t/general-log-warning-question/338443/7 "2023-07-17T20:38:36Z")
Ok thanks Stephen! Very helpful!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 14, 2023, 10:39pm UTC](https://discuss.elastic.co/t/general-log-warning-question/338443/8 "2023-08-14T22:39:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.