Channeling from amazon S3 to ES - speed

Elastic Stack Elasticsearch
1

Hello!

I'm currently hosting data on an Amazon S3, and am wondering, which one would be the faster, more efficient solution to get my data to Elasticsearch: Channeling through Amazon's Lambda, or with Logstash? Are there any pros/cons for each option?

Thank you!

2

Hi @lifesthateasy,

without knowing too much about your data or anything else in your setup: This depends on so many factors and guessing won't really help you. I suggest you build a concrete (but maybe stripped down) version of each ingestion pipeline and take real measurements.

An obvious disadvantage of Lamba is that this is a proprietary product whereas Logstash is open source (disclaimer: I work for Elastic which also sponsors Logstash development).

Daniel