AWS LB logs are stored in S3 bucket at AWSLogs/xxxxxxxxx/elasticloadbalancing/region/2018/10/16/<log_files>. I want to skip old logs and start parsing from start of this month.
Setting prefix AWSLogs/xxxxxxxxx/elasticloadbalancing/region/2018/10 will temporarily solve my problem but prefix will change at the start of next month. I am using exclude_pattern => "/*dev-test*" to skip dev and test logs. ELK version: 6.4.0
I have written a shell script to update the prefix to current date and set a cron job to execute it every day after midnight and restart logstash.
Please let me know if there is any better solution.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.