Error while creating index template in elastisearch

Hi everyone,

This is my setup :

  • Elasticsearch 7.3.2 + Kibana 7.3.2 on Ubuntu (x-pack/ssl on both)
  • logstash 7.3.2 on Ubuntu

I get the following error while trying to run logstash against ES :

$ sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash
...
[2019-09-30T13:12:22,469][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=> <...cropped...>}}
warning: thread "Ruby-0-Thread-5: :1" terminated with exception (report_on_exception is true):
LogStash::Outputs::Elasticsearch::HttpClient::Pool::BadResponseCodeError: Got response code '403' contacting Elasticsearch at URL 'https://xxx:9200/logstash'
perform_request at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80
perform_request_to_url at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:291
...

I created the logstash_writer role :

curl -k -u elastic:xxx -X GET "https://xxx:9200/_security/role/logstash_writer"
{
"logstash_writer":
{
"cluster": ["manage_index_templates","monitor","manage_ilm"],
"indices": [
{
"names":["logstash-*"],
"privileges":["write","create","delete","create_index","manage","manage_ilm"],
"allow_restricted_indices":false
}],
"applications":,
"run_as":,
"metadata":{},
"transient_metadata":{"enabled":true}
}
}

I also created the logstash_internal :

curl -k -u elastic:xxx -X GET "https://xxx:9200/_security/user/logstash_internal"
{
"logstash_internal":
{
"username":"logstash_internal",
"roles":["logstash_writer"],
"full_name":"Internal Logstash User",
"email":null,
"metadata":{},
"enabled":true
}
}

My /etc/logstash/logstash.yml is :

path.data: /var/lib/logstash
path.logs: /var/log/logstash

My conf.d/pipeline-filebeat.conf file looks like :

input {
beats {
port => "5044"
}
}
output {
elasticsearch {
hosts => [ "xxx:9200" ]
ssl => true
cacert => '/etc/logstash/certs/elastic_ca.crt'
user => logstash_internal
password => 'xxx'
index => "logstash-%{+YYYY.MM.dd}"
}
}

When I try curl -k -u logstash_internal:xxx -XPUT "https://xxx:9200/logstash-idxtest2", I receive :

{
"acknowledged":true,
"shards_acknowledged":true,
"index":"logstash-idxtest2"
}

I guess 'logstash_internal' has proper rights on logstash-* indexes, yet, I get the error shown above when I try to start logstash.

I think i'm close to the solution, but I can't find what I'm missing... Sorry if this question has already been answered, I could not find a proper solution.

Thanks in advance,
Dj

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.