AWS ElasticSearch Service MetricBeats

Elastic Stack Beats
1

I am trying to set up MetricBeats with AWS hosted ElasticSearch service. However, in the metricbeats.yml file it asks for a user name and password.. and there isn't a user name and password in AWS ElasticSearch/Kibana for sending data. I have it configured to accept data from every where unauthenticated. Metricbeats is running on my logstash server which is sending data to the same ES cluster with no issues. This is my error message. Please advise.

Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch https://vpc-chzd52f2bk2ay7y5p2fu.us-east-1.es.amazonaws.com:443: Connection marked as failed because the onConnect callback failed: cannot retrieve the elasticsearch license: unauthorized access, could not connect to the xpack endpoint, verify your credentials]

1 Like
2

Hey @vennemp,

I think that AWS Elasticsearch only includes the Apache licensed features (you can read more about this here). Default Metricbeat packages need some "x-pack" endpoints that are only available in the default Elasticsearch distribution (with Elastic License).
This error maybe related to the lack of this xpack endpoints and not to authentication issues. You may need to use the OSS distribution of Metricbeat, available here: https://www.elastic.co/downloads/beats/metricbeat-oss

3

That makes sense. I knew the AWS flavor of ES/Kibana had some lovely fine print. Unfortunately, I read the Metricbeat OSS version did not support the AWS metrics plug in. If you can confirm that is not the case I would love to implement this.

4

Yeah, metricbeat OSS doesn't support most of the cloud monitoring features.

5

Thanks Jaime.. What I don't understand is since Metricbeat AWS Plugin isn't supported in the AWS-flavor of ES, how did I ever get the pre-configured dashboards?? The command metricbeat setup always failed saying I need a user name pw or the x-pack. But clearly the dashboards appeared at some point.. Is there another way they could have appeared?

6

Not sure, maybe metricebeat setup --dashboards worked at some point?
In any case I am not sure if data ingestion will work... only Apache Licensed Metricbeat should be used with Apache Licensed Elasticsearch.

7

The data ingestion didn't work out of the box, but i sent it to logstash and used logstash to send to ES. So that works

Capture
Capture1971×743 69.2 KB

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.