What should be specified in the filebeats config to connect to Elastic search cloud?

Elastic Stack Beats
1

I am trying to setup filebeats to connect to elasticsearch cloud to send my custom application .json log file.

What configuration setting do I need to add to the filebeat.yml config to send logs to the correct elasticsearch cloud instance ?

Running a

-> % ./filebeat test output
elasticsearch: https://jcc-firstdeployment.kb.us-central1.gcp.cloud.es.io:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 35.193.143.25
    dial up... ERROR dial tcp 35.193.143.25:9200: connect: operation timed out

checking config with filebeat test config - is OK.

I believe to be pointing this to the incorrect cloud location.... can someone assist?

2

I do not use Elastic Cloud, but if I'm not wrong this endpoint is for Kibana, it as a kb , you need to use the endpoint to Elasticsearch.

I think you just need to replace the kb by es.

1 Like
3

Thank you, unfortunately, changing from "kb" to "es" results in the same error:


elasticsearch: https://jcc-firstdeployment.es.us-central1.gcp.cloud.es.io:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 35.193.143.25
    dial up... ERROR dial tcp 35.193.143.25:9200: connect: operation timed out
4

That is correct... BUT you need to add the port :443 otherwise it will default to 9200, which is not correct (that should be in the docs)

https://jcc-firstdeployment.es.us-central1.gcp.cloud.es.io:443

cloud.id will work as is

Screenshot 2024-01-04 at 9.36.40 AM
Screenshot 2024-01-04 at 9.36.40 AM2830×1282 320 KB

5

Looks like that might have done it, thank you.
I have a different problem now:

Error:

talk to server... ERROR failed to parse JSON response: invalid character '<' looking for beginning of value

however I will work off of these new errors now and circle back via a separate thread if I hit a wall

Thank you for your help!

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.