# Need to define a role for Filebet and metricbeat for writing to elasticsearch

**URL:** <https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503>\
**Category:** Beats\
**Tags:** elastic-stack-security, filebeat, metricbeat\
**Created:** [May 9, 2021, 1:24pm UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503 "2021-05-09T13:24:16Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![mhare](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhare/32/52213_2.png) [@mhare](https://discuss.elastic.co/u/mhare)\
**Post date:** [May 9, 2021, 1:24pm UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/1 "2021-05-09T13:24:16Z")

</div>

I am running v7.7 of the stack from a ZIP install on Windows.  
But I have the same issue with v7.12 running as docker container on a Mac.

I've searched and searched and either I don't know what to search for, or I don't recognize the answer when I see it.

I have a pair of ES nodes, one for ingest, hot, data the other for warm when using ILM. I am sending data from a filebeat and a metricbeat instance. I can create user, use it in the filebeat.yml and give is some pretty open privileges (like superuser) and it writes the data just fine. But there must be a better defined role for doing this without such open access. So, the question is, what are the best role privileges to assign a user for filebeat and for metricbeat?

Thanks for the help!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 9, 2021, 1:31pm UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/2 "2021-05-09T13:31:48Z")

</div>

Did you see this?

> **[Grant users access to secured resources | Filebeat Reference \[7.12\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/feature-roles.html)**

I kinda like the API method nice example here, defines in detail what privileges filebeat needs.

> **[Grant access using API keys | Filebeat Reference \[7.12\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/beats-api-keys.html)**

You can add multiple roles to the API key like monitor and writer so it can be used for both.

---

<div class="post-metadata">

**Author:** ![mhare](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhare/32/52213_2.png) [@mhare](https://discuss.elastic.co/u/mhare)\
**Post date:** [May 9, 2021, 1:41pm UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/3 "2021-05-09T13:41:11Z")

</div>

I did. I followed the part for "Grant privileges and roles needed for publishing" (I assumed Filebeat was 'publishing' log data to ES) and I do not ingest any data. This is what prompted my question here.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 9, 2021, 2:02pm UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/4 "2021-05-09T14:02:04Z")

</div>

Ahh... Perhaps provide a the code of the role you defined and did you look to see if there are any errors in the filebeat log?

I just used the API key [here](https://www.elastic.co/guide/en/beats/filebeat/current/beats-api-keys.html#beats-api-key-publish) pretty much verbatim and it worked for writing

```
POST /_security/api_key
{
  "name": "filebeat_host001", 
  "role_descriptors": {
    "filebeat_writer": { 
      "cluster": ["monitor", "read_ilm"],
      "index": [
        {
          "names": ["filebeat-*"],
          "privileges": ["view_index_metadata", "create_doc"]
        }
      ]
    }
  }
}

```

If you changes any of the index names etc... that will matter.

Pretty sure this will work for writing and monitoring...

```
POST /_security/api_key
{
  "name": "filebeat_host001",
  "role_descriptors": {
    "filebeat_writer": {
      "cluster": [
        "monitor",
        "read_ilm"
      ],
      "index": [
        {
          "names": [
            "filebeat-*"
          ],
          "privileges": [
            "view_index_metadata",
            "create_doc"
          ]
        }
      ]
    },
    "filebeat_monitoring": {
      "cluster": [
        "monitor"
      ],
      "index": [
        {
          "names": [
            ".monitoring-beats-*"
          ],
          "privileges": [
            "create_index",
            "create"
          ]
        }
      ]
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![mhare](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhare/32/52213_2.png) [@mhare](https://discuss.elastic.co/u/mhare)\
**Post date:** [May 9, 2021, 5:32pm UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/5 "2021-05-09T17:32:30Z")

</div>

We don't have TLS/HTTPS, so no API keys for us

This is my Role:

```auto
{
      "fb_svc_new" : {
        "cluster" : [
          "monitor",
          "read_ilm"
        ],
        "indices" : [
          {
            "names" : [
              "filebeat-*"
            ],
            "privileges" : [
              "create_doc",
              "view_index_metadata"
            ],
            "allow_restricted_indices" : false
          }
        ],
        "applications" : [],
        "run_as" : [],
        "metadata" : { },
        "transient_metadata" : {
          "enabled" : true
        }
      }
    }

```

and, I do have an ERROR in the filebeat log

```auto
2021-05-09T12:20:45.661-0500	ERROR	[publisher_pipeline_output]	pipeline/output.go:106	Failed to connect to backoff(elasticsearch(http://localhost:9200)): Connection marked as failed because the onConnect callback failed: failed to create alias: {"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:admin/create] is unauthorized for user [fb_service]"}],"type":"security_exception","reason":"action [indices:admin/create] is unauthorized for user [fb_service]"},"status":403}: 403 Forbidden: {"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:admin/create] is unauthorized for user [fb_service]"}],"type":"security_exception","reason":"action [indices:admin/create] is unauthorized for user [fb_service]"},"status":403}

```

This seems to indicate I need more privileges?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 9, 2021, 7:20pm UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/6 "2021-05-09T19:20:00Z")

</div>

Looks like perhaps you need the [setup privileges](https://www.elastic.co/guide/en/beats/filebeat/current/privileges-to-setup-beats.html) as well not sure if you already ran setup separately.

Looks like it's trying to create the alias

---

<div class="post-metadata">

**Author:** ![mhare](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhare/32/52213_2.png) [@mhare](https://discuss.elastic.co/u/mhare)\
**Post date:** [May 9, 2021, 10:08pm UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/7 "2021-05-09T22:08:54Z")

</div>

ok, adding the setup privileges to the publisher privileges is working. So, to recap.. my role now looks like this:

```auto
  "fb_svc_new" : {
    "cluster" : [
      "monitor",
      "read_ilm"
    ],
    "indices" : [
      {
        "names" : [
          "filebeat-*"
        ],
        "privileges" : [
          "create_doc",
          "view_index_metadata",
          "create_index",
          "create",
          "manage",
          "read",
          "write"
        ],
        "allow_restricted_indices" : false
      }
    ],
    "applications" : [],
    "run_as" : [],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}

```

Correct?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 10, 2021, 12:11am UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/8 "2021-05-10T00:11:54Z")

</div>

Well if that works good, I guess I would have expected `manage_ilm` as defined by the docs I referenced above for setup, but if `read_ilm` works then good.

---

<div class="post-metadata">

**Author:** ![mhare](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhare/32/52213_2.png) [@mhare](https://discuss.elastic.co/u/mhare)\
**Post date:** [May 10, 2021, 12:28am UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/9 "2021-05-10T00:28:26Z")

</div>

Oh.. I missed that in the doc. Probably need to make that change as well.

Thanks for all the help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 7, 2021, 2:28am UTC](https://discuss.elastic.co/t/need-to-define-a-role-for-filebet-and-metricbeat-for-writing-to-elasticsearch/272503/10 "2021-06-07T02:28:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.