Using Logstash to upload to S3 log line by line

Elastic Stack Logstash
1

I am using LogStash to upload log files in a Volatile K8S system currently running on GCP.

I am aiming to upload log files to an AWS s3 bucket using LogStash s3 output plugin. As I understand from the documentation the LogStash s3 output will write received log line into a temporary file and by the user-defined size_file and time_file, will set limits for when LogStash will push the temporary files to the s3 bucket

However due to the nature of the system. It is impossible to correctly predict which time or size to push the temporary files always causing the final log lines not being pushed into the s3 bucket.

So is there to configure the s3 output plugin to push to s3 bucket as soon as it receives a logline
preferably to the same file in the s3 bucket.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.