Hi everyone,
This is my setup :
- Elasticsearch 7.3.2 + Kibana 7.3.2 on Ubuntu (x-pack/ssl on both)
- logstash 7.3.2 on Ubuntu
I get the following error while trying to run logstash against ES :
$ sudo /usr/share/logstash/bin/logstash --path.settings /etc/logstash
...
[2019-09-30T13:12:22,469][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=> <...cropped...>}}
warning: thread "Ruby-0-Thread-5: :1" terminated with exception (report_on_exception is true):
LogStash::Outputs::Elasticsearch::HttpClient::Pool::BadResponseCodeError: Got response code '403' contacting Elasticsearch at URL 'https://xxx:9200/logstash'
perform_request at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80
perform_request_to_url at /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:291
...
I created the logstash_writer role :
curl -k -u elastic:xxx -X GET "https://xxx:9200/_security/role/logstash_writer"
{
"logstash_writer":
{
"cluster": ["manage_index_templates","monitor","manage_ilm"],
"indices": [
{
"names":["logstash-*"],
"privileges":["write","create","delete","create_index","manage","manage_ilm"],
"allow_restricted_indices":false
}],
"applications":,
"run_as":,
"metadata":{},
"transient_metadata":{"enabled":true}
}
}
I also created the logstash_internal :
curl -k -u elastic:xxx -X GET "https://xxx:9200/_security/user/logstash_internal"
{
"logstash_internal":
{
"username":"logstash_internal",
"roles":["logstash_writer"],
"full_name":"Internal Logstash User",
"email":null,
"metadata":{},
"enabled":true
}
}
My /etc/logstash/logstash.yml is :
path.data: /var/lib/logstash
path.logs: /var/log/logstash
My conf.d/pipeline-filebeat.conf file looks like :
input {
beats {
port => "5044"
}
}
output {
elasticsearch {
hosts => [ "xxx:9200" ]
ssl => true
cacert => '/etc/logstash/certs/elastic_ca.crt'
user => logstash_internal
password => 'xxx'
index => "logstash-%{+YYYY.MM.dd}"
}
}
When I try curl -k -u logstash_internal:xxx -XPUT "https://xxx:9200/logstash-idxtest2", I receive :
{
"acknowledged":true,
"shards_acknowledged":true,
"index":"logstash-idxtest2"
}
I guess 'logstash_internal' has proper rights on logstash-* indexes, yet, I get the error shown above when I try to start logstash.
I think i'm close to the solution, but I can't find what I'm missing... Sorry if this question has already been answered, I could not find a proper solution.
Thanks in advance,
Dj