Hi all.
I have elastic cloud 7.10.2 and we are setting up logstash with a pipilne with input/filter/output. Also we are using helm to deploy.
Of course I have cloud.id and cloud.auth and not sure how to configure the outut section with those 2 field.
is it the below snippet correct:
output {
elasticsearch {
cloud_id => "MyID:KEY"
cloud_auth => "username:password"
}
}
Cheers
Here are the docs and plugin config. Looks like you are doing it correctly.
cloud_id <label>:<cloud-id>
cloud_auth <username>:<password>
Thanks heaps Aaron. But when I run the logstash I got the following error:
Blockquote
warning: thread "Ruby-0-Thread-6: :1" terminated with exception (report_on_exception is true):
LogStash::Outputs::Elasticsearch::HttpClient::Pool::BadResponseCodeError: Got response code '403' contacting Elasticsearch at URL 'ESURL:443/logstash'
Then another error:
Blockquote
[2021-02-09T09:06:22,973][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Outputs::Elasticsearch::HttpClient::Pool::BadResponseCodeError: LogStash::Outputs::Elasticsearch::HttpClient::Pool::BadResponseCodeError>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:80:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:332:in `perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:319:in `block in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:414:in `with_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:318:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:326:in `block in Pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/http_client.rb:341:in `exists?'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/http_client.rb:363:in `rollover_alias_exists?'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/ilm.rb:91:in `maybe_create_rollover_alias'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/ilm.rb:10:in `setup_ilm'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.3-java/lib/logstash/outputs/elasticsearch/common.rb:50:in `block in setup_after_successful_connection'"]}
Any idea?
Cheers
You are getting your cloud ID from here?
Using `elastic:password? Try resetting password?
The URL ESURL:443/logstash which it tried to reach out looks wrong. Are you using GCP, Azure, or AWS?
Hi Aaron
ESURL:443/logstash is correct as I didn't want to expose my actual URL.
In the meantime I opened (all) the credentials for that user for the logstash and it's working fine but of course I'd rather give specific permission for that and all opened.
Is there a role/permission for logstash?
Cheers
I just tested this out of the box and didn't change any settings/roles.
If you copy the ES endpoint and put it in the browser, enter user/pass, do you get a response? This is just testing authentication without Logstash.
all the endpoint elastic and kibana works perfectly. I deployed filebeat and metricbeat with no problems at all. With logstash I am using (like for file and metricbeat) a different user and not the user I use to login onto Kibana.
but I am having issue with the permission only for logstash as I have a pipeline with input/filter and output.
The rest works fine.
Do you have any Traffic Filters set?
no traffic filters!
What I am looking for is the right permissions for a user created only for all the beats I ve been using (file,metric). For logstash I need special permission which I am still looking for.
The same user (not the cloud/root user) works fine with the metricbeat and filebeat... but not with logstash so I guess I need different permissions
Cheers
Using your cloud user/password should have the correct permissions and works out of the box. If this doesn't work I would open an issue on GitHub.
Construct this value by following this format ":". Use your Cloud username for the first part. Use your Cloud password for the second part, which is given once in the Cloud UI when you create a cluster.
If you want to use a created user/roles instead of elastic then you need to configure those.
thanks Aaron. Unfortunately if I give only the logstash permission, I got the same error.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
