# Authentication using apikey failed

**URL:** <https://discuss.elastic.co/t/authentication-using-apikey-failed/333244>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [May 12, 2023, 2:23am UTC](https://discuss.elastic.co/t/authentication-using-apikey-failed/333244 "2023-05-12T02:23:26Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmrlixos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmrlixos/32/99836_2.png) [@dmrlixos](https://discuss.elastic.co/u/dmrlixos)\
**Post date:** [May 12, 2023, 2:23am UTC](https://discuss.elastic.co/t/authentication-using-apikey-failed/333244/1 "2023-05-12T02:23:26Z")

</div>

Hi

I have a cluster with 3 instances ( 1 Master 2 Data Nodes )  
Recenty looking into my cluster, i found a lot of warnings about Authentication using apikey failed on specific apikey id EjkscocB14\*\*\*\*\*\*\*\*

I try search more about this apikey, but she not present on my api keys, probabily the reason of the error.

How i find who are using this api key ?

I using fleet with some integrations like windows,iis, nginx.

How resolve this warn error ?  
Is possible do this ? Someone can help me ?

Thks

---

<div class="post-metadata">

**Author:** ![dmrlixos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmrlixos/32/99836_2.png) [@dmrlixos](https://discuss.elastic.co/u/dmrlixos)\
**Post date:** [May 22, 2023, 12:58am UTC](https://discuss.elastic.co/t/authentication-using-apikey-failed/333244/2 "2023-05-22T00:58:11Z")

</div>

someone can help, or have some idea to fix this?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 22, 2023, 3:19am UTC](https://discuss.elastic.co/t/authentication-using-apikey-failed/333244/3 "2023-05-22T03:19:52Z")

</div>

Hi @dmrlixos

Apologies but there is not much to go on.

Can you share the entire authentication message?

You can turn up regular logging to to get more details ser [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/logging.html#configuring-logging-levels)

Probably set

```auto
PUT /_cluster/settings
{
  "persistent": {
    "logger.org.elasticsearch.security": "DEBUG"
  }
}

```

Or perhaps

`"logger.org.elasticsearch.security.authc": "DEBUG"`

If you have a commercial subscription you turn in audit logging.

Yes this may have to do with fleet or an agent that was not updated/ properly removed etc... So the API Key is out of date, you can look on the source side.

You need to dig in and / or provide more detail

---

<div class="post-metadata">

**Author:** ![dmrlixos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmrlixos/32/99836_2.png) [@dmrlixos](https://discuss.elastic.co/u/dmrlixos)\
**Post date:** [May 23, 2023, 2:25am UTC](https://discuss.elastic.co/t/authentication-using-apikey-failed/333244/4 "2023-05-23T02:25:07Z")

</div>

Hi @stephenb

About the full message WARN.

```auto
May 23, 2023 2:10:46 AM WARN Server Authentication using apikey failed - unable to find apikey with id EjkscocB1498qJ03nQl***

```

Environment:  
`i'm using fleet with some integrations like windows,linux,iis to get theses logs by elastic agents.`

I think too, that this message it's about some fleet agent that was be uninstalled incorrect, your explain make sense, but I don't have idea how agent is, I just have the api key id.

Subscription  
` I'm using a free and the audit logs is not allowed.`

If want more details, please tell me

Thanks.

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [May 23, 2023, 2:51am UTC](https://discuss.elastic.co/t/authentication-using-apikey-failed/333244/5 "2023-05-23T02:51:22Z")

</div>

> [@dmrlixos](#):
>
> `I'm using a free and the audit logs is not allowed.`

In that case, you can try HttpTracer which will report the remote address of each request.

```auto
PUT _cluster/settings
{
   "persistent" : {
      "logger.org.elasticsearch.http.HttpTracer" : "TRACE"
   }
}

```

The logs will get quite verbose. You should see something like the follows for the failed API key authentication

> received request from [Netty4HttpChannel{localAddress=/[0:0:0:0:0:0:0:1]:9200, remoteAddress=/[0:0:0:0:0:0:0:1]:53497}]org.elasticsearch.http.HttpHeadersValidationException: org.elasticsearch.ElasticsearchSecurityException: unable to authenticate with provided credentials and anonymous access is not allowed for this request

Hopefully the remote address will give you enough information to identify the agent that is using the invalid API key. Once you are done, you can remove HTTP tracer logging with

```auto
PUT _cluster/settings
{
   "persistent" : {
      "logger.org.elasticsearch.http.HttpTracer" : null
   }
}

```

---

<div class="post-metadata">

**Author:** ![dmrlixos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dmrlixos/32/99836_2.png) [@dmrlixos](https://discuss.elastic.co/u/dmrlixos)\
**Post date:** [May 23, 2023, 2:08pm UTC](https://discuss.elastic.co/t/authentication-using-apikey-failed/333244/6 "2023-05-23T14:08:21Z")

</div>

Hi @Yang_Wang

I enabled the trace http, but the error still are displayed like bellow

> Authentication using apikey failed - unable to find apikey with id EjkscocB1498qJ03nQ\*\*\*

The trace not show the full request, but from others requests are displayed fine, like this.

> [null][OK][application/json][547] sent response to [Netty4HttpChannel{localAddress=/10._._._:9200, remoteAddress=/172._.\*.\*1:18540}] success [true]

this

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 20, 2023, 2:08pm UTC](https://discuss.elastic.co/t/authentication-using-apikey-failed/333244/7 "2023-06-20T14:08:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
