Winlogbeat API Key Permissions

[Coinology (Joshua Jones)]

All,

I'm hoping someone can help. I used to output data from Winlogbeat to Logstash and index from there, everything worked fine. With 7.9 I attempted to use Elasticsearch output and received the below:

WARN [elasticsearch] elasticsearch/client.go:407 Cannot index event publisher. ... (status=403): {"type":"security_exception","reason":"action [indices:admin/auto_create] is unauthorized for API key id [X] of user [X]

My Winlogbeat config is as follows:

setup.ilm.check_exists: false
setup.template.settings:
  index.number_of_shards: 1
output.elasticsearch:
  hosts: ["https://ES1:9200"]
  api_key: "${key}"
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
monitoring:
  enabled: true
  cluster_uuid: UjKmT6nUSuxItYO5eHSy3g
  elasticsearch:
    hosts: ["https://ES1:9200"]
    api_key: "${key}"

Note that I am using the same API key for both my monitoring and publishing currently since both are going to the same cluster. API key privileges I used from this doc and are as follows:

{
    "name": "winlogbeat_host00001", 
    "role_descriptors": {
        "winlogbeat_monitoring": { 
            "cluster": ["monitor"],
            "index": [
            {
                "names": [".monitoring-beats-*"],
                "privileges": ["create_index", "create"]
            }
            ]
        },
        "winlogbeat_writer": { 
            "cluster": ["monitor", "read_ilm"],
            "index": [
            {
                "names": ["winlogbeat-*"],
                "privileges": ["view_index_metadata", "create_doc"]
            }
            ]
        }
    }
}

I'm bound to be missing something silly. Anyone have any ideas?? Thank you!

[shaunak (Shaunak Kashyap)]

Hi @Coinology, just to help narrow this down a bit, when you were using the Logstash output (output.logstash:) previously, did you still have the exact same monitoring: configuration section?

Thanks,

Shaunak

[Coinology (Joshua Jones)]

@shaunak I did indeed have the exam same monitoring config. I have a feeling this is something to do with ILM, but I am having trouble narrowing it down.

[Coinology (Joshua Jones)]

I decided to throw out the API key (temporarily) and use an account with more privileges just long enough to see what it was trying to do.

It appears it is creating an index named Winlogbeat-7.9.1 and writing to that rather than doing something like Winlogbeat-7.9.1-000001 with a write alias. Is this expected behavior?

Just a note: I've created a setup role as outlined here and ran .\winlogbeat.exe setup -e with it to set up the index template, aliases, etc.

I'm really not sure what [indices:admin/auto_create] is. I don't see such a privilege defined here. I'm at a loss on this one. I believe I am confusing myself more the more I look into it.

I may be able to just bootstrap an index with write alias and have Winlogbeat just write to the write alias, but isn't this all supposed to be handled by Winlogbeat now?

[Coinology (Joshua Jones)]

I've resolved this. I'm not sure exactly where the issue was, but this is what I did:

1. Removed setup.ilm.check_exists: false from Winlogbeat.yml
2. Added setup.ilm.rollover_alias: "winlogbeat" to Winlogbeat.yml
3. Re-ran .\winlogbeat.exe setup -e
4. Adjusted my API key publishing role to point to winlogbeat* index rather than winlogbeat-* (needed for "winlogbeat" rollover_alias)
5. Changed create privilege on monitoring role to create_doc (I don't believe this did anything for this particular issue, but I noticed there was a conflict between this doc and this doc - I changed it to match the latter since it seems more restrictive)

I'm pretty sure I had ILM misconfigured with setup.ilm.check_exists set to false, although I'm not 100%. The docs aren't very clear.