Collecting logs from Azure EH

[wedkarz014 (Patryk)]

Hello,
My question is, which tool should i use to collect data from Eh, logstash: Azure Event Hubs plugin | Logstash Reference [8.8] | Elastic or filebeat: Azure eventhub input | Filebeat Reference [8.8] | Elastic ? Which solution is better and why?

Best
Patryk

[leandrojmp (Leandro Pereira)]

@wedkarz014

Which solution is better and why?

Only you can say as it depends entirely on your use case.

Logstash is more flexible and powerful than Filebeat, but it also needs more resources since it runs on a JVM.

Filebeat is more lightweight, but has some limitations if you need to transform your data and can only have one output.

Normally you would use Logstash if you want to transform your data with multiple filters and enrich or if you need to send it to multiple places at the same time.

While today you can use Ingest Pipelines in Elasticsearch to transform and enrich your data, it is pretty limited compared to Logstash, but this works for many cases, you just need to test to see if you what you want to do can be done without Logstash or not.