Custom logs from Logstash to Cloudwatch

Elastic Stack Logstash
1

Hiya,

Currently there is no actual support to send custom logs from Logstash to Cloudwatch. (There is a way to send metrics data using the Cloudwatch Output Plugin) and there was some support for a plugin called logstash-output-cloudwatchlogs, GitHub - amazon-archives/logstash-output-cloudwatchlogs: A logstash plugin that allows to send logs to AWS CloudWatch Logs service.

But it doesn't work anymore. I tried sending the logs from filebeat but it can only output to one destination.

Now, I'm trying to get it sent from Logstash as the flow is Filebeat -> Logstash -> Elasticsearch.

Is there any possibility to make this work in Logstash or can I send the logs from Elasticsearch to Cloudwatch?

2

Send the logs from filebeat to cloudwatch? Filbeat does not have an output for Cloudwatch, the only available outputs are these.

Unfortunately no for both cases, Logstash does not have an output plugin to Cloudwatch and Elasticsearch does not send data to anywhere.

An alternative would be to use a different tool, like vector.

In this case you would need to change your data ingestion flow and add a Kafka to make it possible to have two different tools reading from the same source.

For example, you would have filebeat sending the logs to Kafka and then both Logstash and vector would read from the same topic, but with different group id, and send the logs to their destinations.

Somethinkg like this:

Filebeat -> Kafka
Kafka -> Logstash -> Elasticsearch
Kafka -> Vector -> Cloudwatch

Depending on what transformations you are doing in Logstash, you may be able to do the same transformations in vector and use vector to ship the logs to both Elasticsearch and Cloudwatch, this way you would be able to drop Logstash.

In my experience you can replace Logstash with Vector in most of the cases.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.