Logstash writer permissions

Elastic Stack Logstash
1

On a new on-premises 8.6 logstash+elasticsearch deployment, I have the following error when configuring my "logstash_writer" role as explained in Secure your connection to Elasticsearch | Logstash Reference [8.6] | Elastic :

[ERROR] 2023-03-06 15:24:14.392 [[main]>worker0] elasticsearch - Elasticsearch setup did not complete normally, please review previously logged errors {:message=>"Got response code '403' contacting Elasticsearch at URL 'https://127.0.0.1:9200/ecs-logstash'", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError}

Giving the "superuser" role to the "logstash_writer" user solves the problem, but is a little bit over the top :wink:

Most probably, I'm missing a permission.

So my two questions :

  • What are the correct permissions for the logstash_writer role ?
  • How to troubleshoot permissions issues ? (elasticsearch logs doesn't show the attempts from logstash)

All insights are welcome !

The logstash config is :

input {
  tcp {
    port => 1514
    type => syslog
    ecs_compatibility => v8
  }
  udp {
    port => 1514
    type => syslog
    ecs_compatibility => v8
  }
}

filter {
  mutate {
    remove_field => [ "type", "message" ]
  }
}

output {
  elasticsearch {
    hosts => ['https://127.0.0.1:9200']
    cacert => '/etc/elasticsearch/certs/http_ca.crt'
    user => logstash_internal
    password => ...
    data_stream => false
    ecs_compatibility => v8
    manage_template => false
  }
  stdout { codec => rubydebug }
}

The user and roles are configured as follow :

{
  "logstash_writer" : {
    "cluster" : [
      "manage_index_templates",
      "monitor",
      "manage_ilm"
    ],
    "indices" : [
      {
        "names" : [
          "ecs-logstash-*"
        ],
        "privileges" : [
          "write",
          "create",
          "create_index",
          "manage",
          "manage_ilm"
        ],
        "allow_restricted_indices" : false
      }
    ],
    "applications" : [ ],
    "run_as" : [ ],
    "metadata" : { },
    "transient_metadata" : {
      "enabled" : true
    }
  }
}
{
  "logstash_internal" : {
    "username" : "logstash_internal",
    "roles" : [
      "logstash_writer"
    ],
    "full_name" : "Internal Logstash User",
    "email" : null,
    "metadata" : { },
    "enabled" : true
  }
}
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.