Security audit index shows tons of fake failed authentications/min (5.1.1) - SOLVED

Elastic Stack Elasticsearch
1

Hi all,

I have x-pack security audit enabled with file and index output. After I set up TLS for inter-node, Kibana and Filebeat communications, security_audit_log* indexes started being filled by hundreds of authentication_failed events for different users (personal accounts of Kibana users, filebeat's etc), whereas ES *_access.log files show only rare real failed logins. There're no such failed authentications and even no events with the same timestamp in logfiles. And these simply don't look real - I definitely don't fail to login by my personal account 100s of times per minute.

All those indexed failed authentication events happen only on 1 of 3 nodes (not currently master) where Kibana is running and pointed to and filebeat is sending logs to. Failures occur only during periods of user activity - e.g. auth failures for my personal account during work hours and filebeat's all the time (live log stream). For personal accounts failures node address and origin address are the same (that one node producing this docs), for filebeat's - origin address is filebeat host's IP.

Realms config is default (so should be native+file).

Environment ES + Kibana + Filebeat, all V5.1 on RHEL7.

Has anyone experienced this?

2

I suspect these are actually realm_authentication_failed events rather than regular authentication_failed events. See https://www.elastic.co/guide/en/x-pack/5.1/auditing.html

If you are using default configuration, then the order tried will be

  • reserved
  • native
  • file

The reserved realm handles builtin users like the elastic superuser.

So, regular native users always generate a realm_authentication_failed event when they log in because they are not authenticated by the reserved realm, and file users will generate two events.

It sounds like you are not interested in the realm_authentication_failed and should put it in the .exclude setting for your audit log configuration

Note: You didn't indicate which patch version of 5.1 you are running. There was a bug that was fixed in 5.1.2 that would cause realm_authentication_failed event to be written to the index-audit-log as if they were authentication_failed events. So if you're not on 5.1.2 you ought to look at upgrading.

3

Thanks a lot, Tim!
The bug is exactly what was happening. Just upgraded to 5.2.0 (was 5.1.1) and excluded the event from logging. Happy now! :slight_smile:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.