How to handle rotating directory in input file for logstash

[selflabs]

Hi Team,

I wants logstash to read logs from AWS S3 bucket but my log path directory is keep rotating on month basis and am not understand how to handle it. please suggest.

input {
      s3 {
          "access_key_id" => "***"
          "secret_access_key" => "***"
          "bucket" => "org"
          "prefix" => "org_2022/ourorg/20220201-20220301/"
          "region" => "us-north-1"
            }
          }

[sholzhauer]

Presuming the 20220201-20220301 part is what rotates it should work by specifying the prefix as
org_2022/ourorg/; It will retrieve all files below it I believe.

[selflabs]

Hi, Yes. "20220201-20220301" is that directory and change on every month basis, under this directory logs are written. like below :
"org_2022/ourorg/20220201-20220301/ourorg.logs".

[sholzhauer]

Then this should work:

input {
      s3 {
          "access_key_id" => "***"
          "secret_access_key" => "***"
          "bucket" => "org"
          "prefix" => "org_2022/ourorg/"
          "region" => "us-north-1"
            }
          }