# HTTP Response 401 "Unauthorized" Using API\_KEY (MetricBeat --\> Kibana)

**URL:** https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462
**Category:** Kibana
**Tags:** elastic-stack-security
**Created:** [May 24, 2022, 7:36am UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462 "2022-05-24T07:36:00Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Lokesh\_Sachdeva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lokesh_sachdeva/32/105782_2.png) [@Lokesh\_Sachdeva](https://discuss.elastic.co/u/Lokesh_Sachdeva)
#### Post date: [May 24, 2022, 7:36am UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462/1 "2022-05-24T07:36:00Z")

</div>

Hi Elastic Team  
We are trying to load data from Metricbeat to Kibana using API KEY. API Key is created and provided full superuser (elastic role) access.

API KEY is able to authenticate into Elasticsearch but while connecting to Kibana during metricbeat setup (.\metricbeat setup -e), it failed with an error : HTTP 401 Unauthorize.

[https://localhost:5601/api/status](https://localhost:5601/api/status) fails: . Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}.

Please note that curl on same URL is working fine with same Authorization Key.

Please find below the Logs and API KEY Access JSON document.

**Setup From MetricBeat:**

```auto
.\Metricbet setup -e
Running Using Metric beat Its failing while loading into Kibana with HTTP 401 Unauthorize Error

2022-05-21T18:14:52.729+0530 INFO instance/beat.go:299 Setup Beat: winlogbeat; Version: 7.10.2
2022-05-21T18:14:52.748+0530 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'winlogbeat-7.10.2' as ILM is enabled.
2022-05-21T18:14:52.765+0530 INFO eslegclient/connection.go:99 elasticsearch url: https://localhost:9200
2022-05-21T18:14:52.771+0530 INFO [publisher] pipeline/module.go:113 Beat name: MMD045048702357
2022-05-21T18:14:52.774+0530 INFO beater/winlogbeat.go:69 State will be read from and persisted to C:\Projects\Data\ELK\Beats\WinLogBeat\data\.winlogbeat.yml
2022-05-21T18:14:53.098+0530 WARN [cfgwarn] registered_domain/registered_domain.go:60 BETA: The registered_domain processor is beta.
2022-05-21T18:14:53.245+0530 WARN [cfgwarn] registered_domain/registered_domain.go:60 BETA: The registered_domain processor is beta.
2022-05-21T18:14:53.266+0530 INFO eslegclient/connection.go:99 elasticsearch url: https://localhost:9200
2022-05-21T18:14:53.364+0530 INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 8.0.0
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

2022-05-21T18:14:53.374+0530 INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
2022-05-21T18:14:53.385+0530 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
2022-05-21T18:14:53.385+0530 INFO [index-management] idxmgmt/std.go:274 ILM policy successfully loaded.
2022-05-21T18:14:53.388+0530 INFO [index-management] idxmgmt/std.go:407 Set setup.template.name to '{winlogbeat-7.10.2 {now/d}-000001}' as ILM is enabled.
2022-05-21T18:14:53.404+0530 INFO [index-management] idxmgmt/std.go:412 Set setup.template.pattern to 'winlogbeat-7.10.2-*' as ILM is enabled.
2022-05-21T18:14:53.410+0530 INFO [index-management] idxmgmt/std.go:446 Set settings.index.lifecycle.rollover_alias in template to {winlogbeat-7.10.2 {now/d}-000001} as ILM is enabled.
2022-05-21T18:14:53.414+0530 INFO [index-management] idxmgmt/std.go:450 Set settings.index.lifecycle.name in template to {winlogbeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2022-05-21T18:14:53.424+0530 INFO template/load.go:183 Existing template will be overwritten, as overwrite is enabled.
2022-05-21T18:14:53.564+0530 INFO template/load.go:117 Try loading template winlogbeat-7.10.2 to Elasticsearch
2022-05-21T18:14:53.908+0530 INFO template/load.go:109 template with name 'winlogbeat-7.10.2' loaded.
2022-05-21T18:14:53.909+0530 INFO [index-management] idxmgmt/std.go:298 Loaded index template.
2022-05-21T18:14:53.918+0530 INFO [index-management] idxmgmt/std.go:309 Write alias successfully generated.
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
2022-05-21T18:14:53.929+0530 INFO kibana/client.go:119 Kibana url: https://localhost:5601
2022-05-21T18:14:53.992+0530 ERROR instance/beat.go:956 Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to https://localhost:5601/api/status fails: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}.
Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to https://localhost:5601/api/status fails: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}.

```

**Connection To Kibana Role**

```auto
Preformatted text`curl -v --cacert "C:\Projects\Certificates\Signer\ca.crt" --location --request GET 'https://localhost:5601/api/security/role' --header 'Content-Type: application/json;charset=UTF-8' --header 'kbn-xsrf: true' --header 'Authorization: ApiKey TU9CSjVvQUJ4YlRRdnExRmJYem86ZUFWS3BmNXFTM0NQc0lYeXIwVEgtQQ=='
Note: Unnecessary use of -X or --request, GET is already inferred.
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Connected to localhost (::1) port 5601 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: C:\Projects\Certificates\Signer\ca.crt
* CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [21 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2018 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=GB; ST=XXXXX; L=XXXXX; O=XXXXX; OU=GI; CN=Server_Certificate
* start date: Apr 29 07:54:26 2022 GMT
* expire date: Apr 28 07:54:26 2024 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: DC=com; DC=XXXXXX; DC=test01global; CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify ok.
} [5 bytes data]
> GET /api/security/role HTTP/1.1
> Host: localhost:5601
> User-Agent: curl/7.78.0
> Accept: */*
> Content-Type: application/json;charset=UTF-8
> kbn-xsrf: true
> Authorization: ApiKey TU9CSjVvQUJ4YlRRdnExRmJYem86ZUFWS3BmNXFTM0NQc0lYeXIwVEgtQQ==
>
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< x-content-type-options: nosniff
< referrer-policy: no-referrer-when-downgrade
< kbn-name: a29bd265d8d8
< kbn-license-sig: a2b93ac49b41ba98f7fefe5376d8a057dea29560144ae157e0ff5e4c42f0d57b
< content-type: application/json; charset=utf-8
< cache-control: private, no-cache, no-store, must-revalidate
< content-length: 17141
< vary: accept-encoding
< accept-ranges: bytes
< Date: Sat, 21 May 2022 12:39:20 GMT
< Connection: keep-alive
< Keep-Alive: timeout=120
<
{ [15918 bytes data]
100 17141 100 17141 0 0 155k 0 --:--:-- --:--:-- --:--:-- 160kcations":[]},{"name":"watcher_admin","metadata":{"_reserved":true},"transient_metadata":{"enabled":true},"elasticsearch":{"cluster":["manage_watcher"],"indices":[{"names":[".watches",".triggered_watches",".watcher-history-*"],"privileges":["read"],"allow_restricted_indices":true}],"run_as":[]},"kibana":[],"_transform_error":[],"_unrecognized_applications":[]},{"name":"watcher_user","metadata":{"_reserved":true},"transient_metadata":{"enabled":true},"elasticsearch":{"cluster":["monitor_watcher"],"indices":[{"names":[".watches"],"privileges":["read"],"allow_restricted_indices":true},{"names":[".watcher-history-*"],"privileges":["read"],"allow_restricted_indices":false}],"run_as":[]},"kibana":[],"_transform_error":[],"_unrecognized_applications":[]}]
* Connection #0 to host localhost left intact

```

**Connection To Elasticsearch:**

```auto
curl -v --cacert "C:\Projects\Data\ELK\Beats\WinLogBeat\certs\ca.crt" --location --request GET 'https://localhost:9200/_security/_authenticate' --header 'Content-Type: application/json;charset=UTF-8' --header 'kbn-xsrf: true' --header 'Authorization: ApiKey TU9CSjVvQUJ4YlRRdnExRmJYem86ZUFWS3BmNXFTM0NQc0lYeXIwVEgtQQ=='

  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying ::1:9200...
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: C:\Projects\Data\ELK\Beats\WinLogBeat\certs\ca.crt
* CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [32 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [877 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=es01
* start date: May 19 10:43:14 2022 GMT
* expire date: May 18 10:43:14 2025 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify ok.
} [5 bytes data]
> GET /_security/_authenticate HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.78.0
> Accept: */*
> Content-Type: application/json;charset=UTF-8
> kbn-xsrf: true
> Authorization: ApiKey TU9CSjVvQUJ4YlRRdnExRmJYem86ZUFWS3BmNXFTM0NQc0lYeXIwVEgtQQ==
>
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [1168 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-elastic-product: Elasticsearch
< content-type: application/json
< content-length: 266
<
{ [266 bytes data]
100 266 100 266 0 0 4019 0 --:--:-- --:--:-- --:--:-- 4222{"username":"elastic","roles":[],"full_name":null,"email":null,"metadata":{"_reserved":true},"enabled":true,"authentication_realm":{"name":"_es_api_key","type":"_es_api_key"},"lookup_realm":{"name":"_es_api_key","type":"_es_api_key"},"authentication_type":"api_key"}
* Connection #0 to host localhost left intact

curl -v --cacert "C:\Projects\Certificates\Signer\ca.crt" --location --request GET 'https://localhost:5601/api/status' --header 'Content-Type: application/json;charset=UTF-8' --header 'kbn-xsrf: true' --header 'Authorization: ApiKey TU9CSjVvQUJ4YlRRdnExRmJYem86ZUFWS3BmNXFTM0NQc0lYeXIwVEgtQQ=='

* Connected to localhost (::1) port 5601 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: C:\Projects\Certificates\Signer\ca.crt
* CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [21 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2018 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=XX; ST=XXXXXXXX; L=XXXXXX; O=XXXXXXXX; OU=GI; CN=ElasticCertificate_client
* start date: Apr 29 07:54:26 2022 GMT
* expire date: Apr 28 07:54:26 2024 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: DC=com; DC=XXXXXXX; DC=test01global; CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify ok.
} [5 bytes data]
> GET /api/status HTTP/1.1
> Host: localhost:5601
> User-Agent: curl/7.78.0
> Accept: */*
> Content-Type: application/json;charset=UTF-8
> kbn-xsrf: true
> Authorization: ApiKey TU9CSjVvQUJ4YlRRdnExRmJYem86ZUFWS3BmNXFTM0NQc0lYeXIwVEgtQQ==
>
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< x-content-type-options: nosniff
< referrer-policy: no-referrer-when-downgrade
< kbn-name: a29bd265d8d8
< kbn-license-sig: a2b93ac49b41ba98f7fefe5376d8a057dea29560144ae157e0ff5e4c42f0d57b
< content-type: application/json; charset=utf-8
< cache-control: private, no-cache, no-store, must-revalidate
< content-length: 12533
< vary: accept-encoding
< accept-ranges: bytes
< Date: Sat, 21 May 2022 12:50:16 GMT
< Connection: keep-alive
< Keep-Alive: timeout=120
<
{ [12533 bytes data]
100 12533 100 12533 0 0 203k 0 --:--:-- --:--:-- --:--:-- 214k1,"95":10.690559,"99":15.335423}},"uptime_in_millis":7601839.708579}],"response_times":{"avg_in_millis":341,"max_in_millis":460},"concurrent_connections":1,"requests":{"disconnects":0,"total":2,"statusCodes":{"200":2},"status_codes":{"200":2}}}}
* Connection #0 to host localhost left intact

```

**Connection To Kibana API Status**

```auto
> curl -v --cacert "C:\Projects\Certificates\Signer\ca.crt" --location --request GET 'https://localhost:5601/api/status' --header 'Content-Type: application/json;charset=UTF-8' --header 'kbn-xsrf: true' --header 'Authorization: ApiKey TU9CSjVvQUJ4YlRRdnExRmJYem86ZUFWS3BmNXFTM0NQc0lYeXIwVEgtQQ=='
> 
> * Connected to localhost (::1) port 5601 (#0)
> 
> * ALPN, offering h2
> 
> * ALPN, offering http/1.1
> 
> * successfully set certificate verify locations:
> 
> * CAfile: C:\Projects\Certificates\Signer\ca.crt
> 
> * CApath: none
> 
> } [5 bytes data]
> 
> * TLSv1.3 (OUT), TLS handshake, Client hello (1):
> 
> } [512 bytes data]
> 
> * TLSv1.3 (IN), TLS handshake, Server hello (2):
> 
> { [122 bytes data]
> 
> * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
> 
> { [21 bytes data]
> 
> * TLSv1.3 (IN), TLS handshake, Certificate (11):
> 
> { [2018 bytes data]
> 
> * TLSv1.3 (IN), TLS handshake, CERT verify (15):
> 
> { [264 bytes data]
> 
> * TLSv1.3 (IN), TLS handshake, Finished (20):
> 
> { [52 bytes data]
> 
> * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
> 
> } [1 bytes data]
> 
> * TLSv1.3 (OUT), TLS handshake, Finished (20):
> 
> } [52 bytes data]
> 
> * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
> 
> * ALPN, server accepted to use http/1.1
> 
> * Server certificate:
> 
> * subject: C=XX; ST=XXXXXXX; L=XXXXX; O=XXXXXXXX; OU=GI; CN=ElasticCertificate_client
> 
> * start date: Apr 29 07:54:26 2022 GMT
> 
> * expire date: Apr 28 07:54:26 2024 GMT
> 
> * subjectAltName: host "localhost" matched cert's "localhost"
> 
> * issuer: DC=com; DC=XXXXXXX; DC=testglobal; CN=Elastic Certificate Tool Autogenerated CA
> 
> * SSL certificate verify ok.
> 
> } [5 bytes data]
> 
> > GET /api/status HTTP/1.1
> 
> > Host: localhost:5601
> 
> > User-Agent: curl/7.78.0
> 
> > Accept: */*
> 
> > Content-Type: application/json;charset=UTF-8
> 
> > kbn-xsrf: true
> 
> > Authorization: ApiKey TU9CSjVvQUJ4YlRRdnExRmJYem86ZUFWS3BmNXFTM0NQc0lYeXIwVEgtQQ==
> 
> >
> 
> { [5 bytes data]
> 
> * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
> 
> { [265 bytes data]
> 
> * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
> 
> { [265 bytes data]
> 
> * old SSL session ID is stale, removing
> 
> { [5 bytes data]
> 
> * Mark bundle as not supporting multiuse
> 
> < HTTP/1.1 200 OK
> 
> < x-content-type-options: nosniff
> 
> < referrer-policy: no-referrer-when-downgrade
> 
> < kbn-name: a29bd265d8d8
> 
> < kbn-license-sig: a2b93ac49b41ba98f7fefe5376d8a057dea29560144ae157e0ff5e4c42f0d57b
> 
> < content-type: application/json; charset=utf-8
> 
> < cache-control: private, no-cache, no-store, must-revalidate
> 
> < content-length: 12533
> 
> < vary: accept-encoding
> 
> < accept-ranges: bytes
> 
> < Date: Sat, 21 May 2022 12:50:16 GMT
> 
> < Connection: keep-alive
> 
> < Keep-Alive: timeout=120
> 
> <
> 
> { [12533 bytes data]
> 
> 100 12533 100 12533 0 0 203k 0 --:--:-- --:--:-- --:--:-- 214k1,"95":10.690559,"99":15.335423}},"uptime_in_millis":7601839.708579}],"response_times":{"avg_in_millis":341,"max_in_millis":460},"concurrent_connections":1,"requests":{"disconnects":0,"total":2,"statusCodes":{"200":2},"status_codes":{"200":2}}}}
> 
> * Connection #0 to host localhost left intact

```

API KEY JSON

```auto
---------------------------API KEY Role Addition------------------------
POST /_security/api_key
{
  "name": "ApiKey_Beats_User",
  "expiration": "365d",
  "role_descriptors": {
      "beats_cluster_access" : {
    "cluster" : [
      "all",
      "grant_api_key",
      "manage",
      "manage_api_key",
      "manage_autoscaling",
      "manage_ccr",
      "manage_data_frame_transforms",
      "delegate_pki",
      "manage_enrich",
      "manage_ilm",
      "manage_index_templates",
      "manage_ingest_pipelines",
      "manage_logstash_pipelines",
      "manage_ml",
      "manage_security"
    ],
    "indices" : [
      {
        "names" : [
          "*"
        ],
        "privileges" : [
          "all",
          "delete",
          "create_index",
          "create_doc",
          "index",
          "maintenance",
          "manage_follow_index",
          "manage_leader_index",
          "manage",
          "read",
          "manage_ilm",
          "auto_configure",
          "create",
          "monitor",
          "view_index_metadata",
          "delete_index",
          "read_cross_cluster",
          "write"
        ],
        "allow_restricted_indices" : false
      }
    ],
    "applications" : [
      {
        "application" : "kibana-.kibana",
        "privileges" : [
          "all"
        ],
        "resources" : [
          "*"
        ]
      }
    ],
    "run_as" : [
      "elastic"
    ],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
  }
}

```

---

<div class="post-metadata">

### Author: ![TonyWRobinson](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@TonyWRobinson](https://discuss.elastic.co/u/TonyWRobinson)
#### Post date: [June 3, 2022, 8:34pm UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462/2 "2022-06-03T20:34:05Z")

</div>

I'd like to here anyone's thoughts on this as well. I want to use api\_key for authentication from metricbeat to the Elasticsearch cluster as well, but thus far the module configuration only seems to work with username and password. We definitely won't put the user name and password in plain text in a config, and I'd really like to avoid having to setup and manage local keystores on the various nodes to store credentials. I did find this issue on GitHub, so would be great if anyone has any updates:

> <https://github.com/elastic/beats/issues/29271>
>
> Metricbeat doesn't work properly with \`api\_key\` in metricbeat elasticsearch modu…le but work ok with \`username\\password\`
> the same \`api\_key\` works fine in the \`output.elasticsearch.api\_key\`
> 
> \`/etc/metricbeat/modules.d/elasticsearch-xpack.yml\`
> \`\`\`yaml
> \---
> \- module: elasticsearch
> xpack.enabled: true
> period: 10s
> hosts: \["https://fqdn:8443"\]
> ssl:
> certificate\_authorities:
> - /etc/ssl/certs/ca-certificates.crt
> api\_key: xxxxxxx:yyyyyyyy
> \`\`\`
> 
> ! note that the same \`api\_key\` works fine in \`/etc/metricbeat/metricbeat.yml\`
> \`\`\`yaml
> \---
> output:
> elasticsearch:
> hosts: \['https://server-01:9200', 'https://server-02:9200', ...\]
> ssl:
> certificate\_authorities: \['/etc/ssl/certs/ca-certificates.crt'\]
> api\_key: xxxxxxx:yyyyyyyy
> 
> metricbeat:
> config:
> modules:
> path: ${path.config}/modules.d/\*.yml
> reload:
> enabled: false
> 
> processors:
> - add\_host\_metadata: ~
> - add\_cloud\_metadata: ~
> - add\_docker\_metadata: ~
> - add\_kubernetes\_metadata: ~
> 
> \`\`\`
> 
> error log:
> \`\`\`syslog
> ....
> Dec 4 21:12:00 elasticsearch-01 metricbeat\[214918\]: 2021-12-04T21:12:00.767Z#011INFO#011module/wrapper.go:259#011Error fetching data for metricset elasticsearch.node\_stats: HTTP error 401 in : 401 Unauthorized
> Dec 4 21:12:10 elasticsearch-01 metricbeat\[214918\]: 2021-12-04T21:12:10.606Z#011INFO#011module/wrapper.go:259#011Error fetching data for metricset elasticsearch.enrich: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized
> Dec 4 21:12:10 elasticsearch-01 metricbeat\[214918\]: 2021-12-04T21:12:10.610Z#011INFO#011module/wrapper.go:259#011Error fetching data for metricset elasticsearch.cluster\_stats: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized
> Dec 4 21:12:10 elasticsearch-01 metricbeat\[214918\]: 2021-12-04T21:12:10.632Z#011INFO#011module/wrapper.go:259#011Error fetching data for metricset elasticsearch.index: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized
> Dec 4 21:12:10 elasticsearch-01 metricbeat\[214918\]: 2021-12-04T21:12:10.660Z#011INFO#011module/wrapper.go:259#011Error fetching data for metricset elasticsearch.shard: error determining if connected Elasticsearch node is master: HTTP error 401 in : 401 Unauthorized
> ....
> \`\`\`
> 
> \- Version: \`metricbeat/stable,now 7.15.2 amd64 \[installed\]\`
> \- Operating System: \`Debian GNU/Linux 11 (bullseye)\`
> 
> solution: use username\\password instead of api\_key in \`/etc/metricbeat/modules.d/elasticsearch-xpack.yml\`

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [June 6, 2022, 4:27am UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462/3 "2022-06-06T04:27:43Z")

</div>

@Lokesh_Sachdeva

Looks like it is failing during setup.. perhaps the setup role is not defined correctly

> **[Grant privileges and roles needed for setup | Metricbeat Reference \[8.2\] |...](https://www.elastic.co/guide/en/beats/metricbeat/current/privileges-to-setup-beats.html)**

Look like perhaps the `kibana_admin` role is missing? (it is not apologies)

---

<div class="post-metadata">

### Author: ![TonyWRobinson](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@TonyWRobinson](https://discuss.elastic.co/u/TonyWRobinson)
#### Post date: [June 6, 2022, 12:55pm UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462/4 "2022-06-06T12:55:11Z")

</div>

Hi, @stephenb When we create an api key, we provided cluster and index privileges. Is there a part of that call where we also can add existing role names? Or do we just include the combination of privileges the kibana\_admin role possesses?

I tried granting ALL cluster privileges to the .metricbeat-\* and metricbeat-\* indexes to the API key I created, and it still didn't work. When I switch to using the remote\_monitoring\_user, it works like a charm, which is what makes me think something is amiss.

To be clear: in my case, this is for the Elasticsearch MODULE. The Elasticsearch output section seems to take the API key without issue.

This is the API we are using to create the api key

> **[Grant access using API keys | Metricbeat Reference \[8.2\] | Elastic](https://www.elastic.co/guide/en/beats/metricbeat/current/beats-api-keys.html)**

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [June 6, 2022, 3:12pm UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462/5 "2022-06-06T15:12:59Z")

</div>

@Lokesh_Sachdeva Please Confirm what version of Elasticsearch and Winlogbeat you are using

It look like perhaps you are using Elasticsearch 8.0.0 and Winlogbeat 7.10.2 is there a reason for that? or is it Metrcibeat (confused title says Metricbeat one thing details say Winlogbeat)

This is very confusing your command says metricbeat but the output is winlogbeat so I do not know what you are actually doing / running.

```auto
.\Metricbet setup -e
Running Using Metric beat Its failing while loading into Kibana with HTTP 401 Unauthorize Error

2022-05-21T18:14:52.729+0530 INFO instance/beat.go:299 Setup Beat: winlogbeat; Version: 7.10.2
2022-05-21T18:14:52.748+0530 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'winlogbeat-7.10.2' as ILM is enabled.
2022-05-21T18:14:52.765+0530 INFO eslegclient/connection.go:99 elasticsearch url: https://localhost:9200
2022-05-21T18:14:52.771+0530 INFO [publisher] pipeline/module.go:113 Beat name: MMD045048702357
2022-05-21T18:14:52.774+0530 INFO beater/winlogbeat.go:69 State will be read from and persisted to C:\Projects\Data\ELK\Beats\WinLogBeat\data\.winlogbeat.yml
2022-05-21

```

There are pretty significant Changes between the 2 versions.

The privileges will need to fit 8.0 or whatever version of Elasticsearch you are running

> **[Grant privileges and roles needed for setup | Metricbeat Reference \[8.0\] |...](https://www.elastic.co/guide/en/beats/metricbeat/8.0/privileges-to-setup-beats.html)**

> **[Grant privileges and roles needed for publishing | Metricbeat Reference \[8.0\]...](https://www.elastic.co/guide/en/beats/metricbeat/8.0/privileges-to-publish-events.html)**

Example I notice for setup role requires `setup_admin`

```auto
GET _security/role/ingest_admin

{
  "ingest_admin" : {
    "cluster" : [
      "manage_index_templates",
      "manage_pipeline" <!--- you have "manage_ingest_pipelines"
    ],
    "indices" : [],
    "applications" : [],
    "run_as" : [],
    "metadata" : {
      "_reserved" : true
    },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}

```

Which does not match up

I would go through the the 2 pages above set the cluster and index privileges... run the GET on each of the roles and set what is needed.

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [June 6, 2022, 4:38pm UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462/6 "2022-06-06T16:38:54Z")

</div>

@Lokesh_Sachdeva

I just went through the Docs (this was 8.2.2)

And the following all worked, I created these while logged in as the `elastic` user.  
This was for metricbeat but should follow same pattern for winlogbeat. ( I have a mac so I can not run winlogbeat)  
I would also try to get your metricbeat and /or winlogbeat and Elasticsearch to the same version.

```auto
# SETUP ONLY 
POST /_security/api_key
{
  "name": "metricbeat-setup",
  "role_descriptors": {
    "metricbeat-setup": {
      "cluster": [
        "monitor",
        "manage_ilm",
        "manage_index_templates",
        "manage_pipeline"
      ],
      "indices": [
        {
          "names": [
            "metricbeat-*"
          ],
          "privileges": [
            "manage"
          ]
        }
      ],
      "applications": [
        {
          "application": "kibana-.kibana",
          "privileges": [
            "all"
          ],
          "resources": [
            "*"
          ]
        }
      ]
    }
  }
}

#PUBLISH ONLY 
POST /_security/api_key
{
  "name": "metricbeat-setup", 
  "role_descriptors": {
    "metricbeat_writer": { 
      "cluster": ["monitor", "read_ilm", "read_pipeline"],
      "index": [
        {
          "names": ["metricbeat-*"],
          "privileges": ["view_index_metadata", "create_doc"]
        }
      ]
    }
  }
}

# SETUP AND PUBLISH
POST /_security/api_key
{
  "name": "metricbeat-setup-publish",
  "role_descriptors": {
    "metricbeat-setup": {
      "cluster": [
        "monitor",
        "manage_ilm",
        "manage_index_templates",
        "manage_pipeline",
        "read_ilm",
        "read_pipeline"
      ],
      "indices": [
        {
          "names": [
            "metricbeat-*"
          ],
          "privileges": [
            "manage",
            "view_index_metadata",
            "create_doc"
          ]
        }
      ],
      "applications": [
        {
          "application": "kibana-.kibana",
          "privileges": [
            "all"
          ],
          "resources": [
            "*"
          ]
        }
      ]
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [June 6, 2022, 4:41pm UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462/7 "2022-06-06T16:41:32Z")

</div>

@TonyWRobinson I think that is a separate issue from OP I would suggest opening a separate topic and be clear (like you are) that you need an API KEY to work with the metricbeat Elasticsearch module.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 4, 2022, 4:41pm UTC](https://discuss.elastic.co/t/http-response-401-unauthorized-using-api-key-metricbeat-kibana/305462/8 "2022-07-04T16:41:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.