Parquet Format Output to S3

Elastic Stack Logstash
1

Is there a way to have Logstash output to an S3 bucket in Parquet format, preferably using Snappy compression?

I see there is an Avro codec but not Parquet one, and a Webhdfs output plugin that allows Snappy compression, but not sure if I can do anything between them and the S3 output plugin to get data into S3 in the particular format I would prefer.

2

So I guess there is no option, and no one else has this use case.

This is unfortunate, since Parquet & ORC are the two columnar data formats recommended by AWS in their guide for improving Athena performance.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.