Events get dropped by restricting api key (winlogbeat & kibana) - Help plz

[Martesch (Martin)]

hello everyone
i need help. this is about the fact that i noticed that in our kibana, which collects our windows event logs certain logs are missing, and this since a certain date, from 31.01.23 to 01.02.23 the amount of eventlogs has halved. and from this time the eventlogs are missing.
Shortly before I changed winlogbeat to API Keys, which are restricted for just publishing.

now i found out through logs from winlogbeat that certain events are dropped, with this message:

Cache:publisher.EventCache{m:mapstr.M(nil)}} (status=403): {"type\":\"security_exception\",\"reason\":\"action [indices:admin/mapping/auto_put] is unauthorized for API key id [XXXXXXXXXXXXX] of user [elastic] on indices [.ds-winlogbeat-8. 5.3-2023.04.01-000005], this action is granted by the index privileges [auto_configure,manage,write,all]\"}, dropping event!", "service.name": "winlogbeat", "ecs.version": "1.6.0"}

until the end of january it worked without problems, but on 01.02.23 a new index was created (every 30 days a new one is created) on which certain events were dropped or only half of the events went through.

Why? Are my API Key restrictions wrong? why did it work until the end of janaur without problems for all events but since the new index in february was created it stopped working?

if i now create a new api key without any restrictions, all events go through again, even the missing ones.

who can help me here to restrict the api key correctly, just for publishing in every index which ist created. i tried with this one Grant access using API keys | Winlogbeat Reference [8.7] | Elastic
but as you see, it didn't work out.

explanation: the eventlogs come from the domain controller and should be saved on an extra server for an IR case, in case of a hack the hacker should of course not be able to delete the logs on the log server with the api key which is used on the DC, so i want to restrict the rights here.

im new to this whole elastic-stack and trying my best
thanks in advance.

[Marius_Dragomir]

Hello Martin,
I have move you issue in the Beats section of the forum as they will have more information on what settings you need for the Winlogbeat API key. Kibana has not influence on this so I cannot offer you much help here.

[andrewkroh]

If this API key is strictly for publishing events from the Beat then I think the it needs to have auto_configure and create_doc privileges. See if that works and we can edit that doc page.

[Martesch (Martin)]

Thanks for your answer, i will try it for one DC and report what happens after the next index rollover.

"id": "XXXXXXXXXXXXXXXXXXXXXX",
      "name": "Winlogbeat_ADGI02",
      "creation": 1683528420038,
      "invalidated": false,
      "username": "elastic",
      "realm": "reserved",
      "metadata": {},
      "role_descriptors": {
        "winlogbeat_writer": {
          "cluster": [
            "monitor",
            "read_ilm",
            "read_pipeline"
          ],
          "indices": [
            {
              "names": [
                "winlogbeat-*"
              ],
              "privileges": [
                "view_index_metadata",
                "create_doc",
                "auto_configure"
              ],
              "allow_restricted_indices": false
            }
          ],
          "applications": [],
          "run_as": [],
          "metadata": {},
          "transient_metadata": {
            "enabled": true
          }
        }
      }
    }
  ]
}

[Martesch (Martin)]

Thanks, it is working now.

[andrewkroh]

Issue logged at [Docs] Incorrect privileges listed in API key documentation · Issue #35612 · elastic/beats · GitHub.