Kibana Dashboard: I am using elastic cloud trial version and filebeat is running but failed to load fllebeat index pattern on kibana

[Manav_Chopra]

I am working on windows system and working on elastic cloud: elasticsearch and kibana
I have installed filebeat as well and configured properly . In console if i run filebeat -e, it is running but on kibana nothing is shown.
No index pattern is coming i.e filebeat-* is expected.

I have enabled the input configuration in filebeat.yml to be true and set the path to C:\Program Files\filebeat\logs*

setup.dashboards.enabled: true
setup.kibana,host: pointed to kibana endpoint url of elastic cloud

Please refer attached images as well and let me know what i am missing?

[ChrsMark (Chris Mark)]

Hey!

For setting up dashboards I explicitly run:
./filebeat setup --dashboards -E setup.kibana.host="https://elastic:xxxxxxxxxx@xxxxxxxxxx.europe-west3.gcp.cloud.es.io:9243" -e -d "*"

Could you give it a shot?

Then I start Filebeat like:
./metricbeat -e -d "*" -E cloud.id=test:xxxxxxxx -E cloud.auth=elastic:xxxxxxx

I don't think that you need output.elasticsearch.hosts, output.elasticsearch.username, output.elasticsearch.password since these are overwritten (https://www.elastic.co/guide/en/beats/metricbeat/current/configure-cloud-id.html#configure-cloud-id).

Hope it helps!

[Manav_Chopra]

Hi there

Thanks for replying. I got into another trouble.
When I ran
./filebeat setup --dashboards -E setup.kibana.host="https://elastic:xxxxxxxxxx@xxxxxxxxxx.europe-west3.gcp.cloud.es.io:9243" -e -d "*"
Exiting: The cloud.id setting enables the Elasticsearch output, but you already have the logstash output enabled in the config

This error is coming.
I have now using logstash as well, so filebeat will take logs and give it to logstash. and logstash will take input from filebeat and then output to elasticsearch

But as you said, i have disabled the settings of output.elasticsearch.hosts , output.elasticsearch.username , output.elasticsearch.password since these are overwritten as I am working on cloud, so above error is coming.
Also if i mentioned and enable the username and password in filebeat.yml file, then different error is coming which is:
Exiting: error loading config file: yaml: line 170: did not find expected key

So please give me solution for both the errors:
a) error loading config file: yaml: line 170: did not find expected key
Please refer https://pastebin.com/UXQa211V for filebeat.yml file
b) Exiting: The cloud.id setting enables the Elasticsearch output, but you already have the logstash output enabled in the config(if i comment the username and password from that yml file, then this error appears)

[ChrsMark (Chris Mark)]

In order to fix issue number 2 you need to remove cloud.* settings from your config since it will try to set Elasticsearch as an output while you already have Logstash to be set. This is what the error indicates:
Exiting: The cloud.id setting enables the Elasticsearch output, but you already have the logstash output enabled in the config

Regarding the first one it seems to be an indentation error, so please try to fix your configuration using an yaml validation tool (you can find many of them online).