Authentication using apikey failed - unable to find apikey with id

[p_vimal (Vimal)]

Hello,

We are running Elasticsearch 7.17.8 and have many error entries like this on in the elastic logs:
[WARN ][o.e.x.s.a.ApiKeyAuthenticator] [NODENAMEE] Authentication using apikey failed - unable to find apikey with id TH6xaoMBpxRmcd35O5Wv

Probable root cause : Our security index got deleted somehow. So we recreated the API keys from scratch but still we are getting this in our logs.

When we try to invalidate/delete the API keys,we couldnt find it in the index.
GET /_security/api_key --> doesnt give the API keys which shows in logs.

We need help in disable this warnings from logs. Can you guide us how to do it?
Regards,
Vimal

[stephenb (Stephen Brown)]

@p_vimal

We are running Elasticsearch 7.17.8 and have many error entries like this on in the elastic logs:
[WARN ][o.e.x.s.a.ApiKeyAuthenticator] [NODENAMEE] Authentication using apikey failed - unable to find apikey with id TH6xaoMBpxRmcd35O5Wv

That looks to me like a client is trying to authenticate using an old / invalid key... did you check that?

[p_vimal (Vimal)]

We dont have any external client accessing our cluster using API keys. Do we have anyway we can find the source IP of this request?

[stephenb (Stephen Brown)]

see here, not easy

Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

Hi @aaronlbk Something does not make sense you have an empty elasticsearch.yml yet you have authentication enabled ... that is not normal / highly unusual (basically not sure how that is even possible) Are you sure that is the elasticsearch.yml that is being used? How did you install elasticsearch? is this an upgrade from 7.x? Normally you would turn on Audit logging but that requires a License. Something is definitely attempting to authenticate... You might be able to see which process…

Also Found This

[ Creating new rule ]: ERROR Authentication using apikey failed - api key has been invalidated

In the current 7.10.x series when a long running rule is still running and you go to update the rule while it is currently running, the API key which represents the rule is invalidated immediately which can result in that rule execution immediately showing an error within that current rule run. The next rule run interval should clear out the error and you should see it run correctly. If on the second rule run it does not clear out and you are consistently seeing this error appear again and agai…