Cannot write Filebeat output to Elastic running on Docker Container on my MAC

Elastic Stack Beats
,
1

I have created a 3 node Elastic Docker Container using the instructions from Elastic 8.2.3, and upgraded it to 8.7.1. I am able to successfully create objects in the Elastic database, and the docker-compose command successfully created the keystores so everything is running correctly as a standalone instance.

I installed Filebeats 8.7.1 on my MAC and using CEL as my filebeat.input and am able to succesfully write the output to the console using output.console. However, when I switch the output to output.elasticsearch, nothing is written to any index on my Docker Elasticsearch. I was able to accomplish this using the 7.17.9 version of Elastic in my Docker Container, but not in the v8. I receive the following error message in the filebeats logs:

{"log.level":"error","@timestamp":"2023-08-02T15:09:00.761-0500","log.logger":"esclientleg","log.origin":{"file.name":"transport/logging.go","file.line":38},"message":"Error dialing x509: “es01” certificate is not standards compliant","service.name":"filebeat","network":"tcp","address":"0.0.0.0:9200","ecs.version":"1.6.0"}

strong text

But I'm not sure if that is causing the error or not. If so, how can I bring my certificate into compliance?

2

Disregard, it turns out Filebeat v8x was populating Elastic via DataStream instead of Index. Everything is working.

3

I went through those steps too. If you don't configure it, it will automatically send to datastream.

4

Does your comment indicate you can configure it to write to an Index? The documentation didn't provide any options to DataStream.

5

Could you show you filebeat.yml ? (without credentials)

6

It turns out that I was expecting the Filebeat output to be in an index as was done in v7x. I have come to realize that in v8x Filebeat now writes to a Data Stream. I was able to find the Data Stream and it is writing as expected now. Thank you for your interest in this.

Marty

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.