Logstash and auto-index creation with ECK

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

I've been looking at replacing our hoomebrew stack with ECK this last week, and have the basic flow from filebeat -> logstash -> elasticsearch working. It's really impressive how it all just wires up together, and the quickstart docs made it a breeze to set up.

I'm now looking at more advanced (relatively) configuration such as controlling the indices used in the logstash output and I don't know whether I'm hitting a bug, trying to do something that's just not supported with ECK, or is just simple user error.

OOTB logstash was writing to an index named logs-generic-default, and I figured to change this all I needed to do was:

      output {
        elasticsearch {
          ...
          index => "mas-%{+YYYY.MM.dd}"
        }
      }

However, this throws up an error because ECK hadn't wired up things such that Logstash can manage indices.

error=>{"type"=>"security_exception", "reason"=>"action [indices:admin/auto_create] is unauthorized for user [eck-mas-eck-mas-logstash-user] with effective roles [eck_logstash_user_role] on indices [mas-2024.05.28], this action is granted by the index privileges [auto_configure,create_index,manage,all]"}}

I assume this is something it can be configured to do, but I have had no success working out how.

When I lookup the mentioned role and user in elasticseaarch (_security/role, _security/user APIs) to see what permissions they had been set up with I get back zero users and 30 roles, none of which are named eck_logstash_user_role. Which is confusing, because it was managing to successfully put the data into that default index.

To get around this I replaced the QS_ES_USER, QS_ES_PASSWORD env var substitutions in the output config with the main elastic username/password and it successfully set up the new index as expected

        elasticsearch {
          hosts => [ "${MAS_ES_HOSTS}" ]
          user => "elastic"
          password => "xxxxxxxxxxxx"
          ssl_certificate_authorities => "${MAS_ES_SSL_CERTIFICATE_AUTHORITY}"
          index => "mas-%{+YYYY.MM.dd}"
        }
      }

Obviously this is just a workaround, I'd like to understand how this is meant to work properly. I couldn't find anything searching these forums, the GitHub issues, or the docs explaining how to set things up such that Logstash would be able to create indexes on-demand, and the missing user & role just added to my confusion.

Any tips/pointers very much welcomed!

2

Hi,

You can create a custom role in Elasticsearch that has the necessary permissions to create indices. The role should have the manage or create_index privilege at the index level.

Create a new logstash_index_writer user.

Once the role is created, you can assign it to the Logstash user.

Regards

3

@durera

Seems you need to extend permissions for the eck_logstash_user_role.
Here is an example how to do that.

Here is my example with extended permissions:

apiVersion: v1
kind: Secret
metadata:
  name: roles
stringData:
  roles.yml: |-
    # Role for ECK Logstash user
    # https://www.elastic.co/docs/deploy-manage/deploy/cloud-on-k8s/configuration-logstash#k8s-logstash-pipelines-es
    eck_logstash_user_role:
      # Default privileges
      cluster: [ 'monitor', 'manage_ilm', 'read_ilm', 'manage_logstash_pipelines', 'manage_index_templates', 'cluster:admin/ingest/pipeline/get']
      indices:
      - names: [ 'logstash', 'logstash-*', 'ecs-logstash', 'ecs-logstash-*', 'logs-*', 'metrics-*', 'synthetics-*', 'traces-*' ]
        privileges: [ 'manage', 'write', 'create_index', 'read', 'view_index_metadata' ]
      # Additional privileges to read and write logs
      - names: ['mycompany-logs*']
        privileges: ['read', 'write', 'view_index_metadata']

---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elasticsearch
spec:
  version: 8.17.0
  auth:
    roles:
      - secretName: roles
  nodeSets:
    - name: default
      count: 3
      config:
        node.store.allow_mmap: false