Logstash S3 input

Elastic Stack Logstash
1

Hi there,
I'm trying to use the logstash S3 input plugin to fetch logs from S3 bucket.

This is my input

s3 {
    id => "terminal_app"
    access_key_id => "${access_key_id}"
    secret_access_key => "${secret_access_key}"
    bucket => "logstash-sb"
    region => "ap-south-1"
    codec => json
    additional_settings => {
          force_path_style => true
          follow_redirects => false
        }
  
  }

This is my output plugin

  else if [@metadata][s3][id] == "terminal_app" {
     elasticsearch {
       hosts => ["localhost:9200"]
       index => "terminal_app_index"
       user => "elastic"
       password => "${es_pwd}"    

     }
   }

I want to write the logs fetched from S3 to a particular index
Is this way that I'm currenty doing the right way because the connection has been made with S3 as I did not encounter any error but there was nothing written in the index that I mentioned in output plugin.

2

Hi,

we can't make assumptions without a full file.

3

What versions of logstash and elasticsearch are you using? In recent versions the index option is ignored because ILM is enabled if supported.

1 Like
4

I found the issue, it was in my conditional statement for S3, I tried with

  else if [type] == "s3"

instead of

else if [@metadata][s3][id] == "terminal_app"

After doing the above change, it was working fine.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.