Get file name from s3 file

Hi
i have s3 bucket from where i taking csv file and uploading in to ES .
i stuck on how to get file name which i have in s3 , beacuse i am using s3 file name as Index name .
My config file -

input{
	s3{
	bucket => "***"
	access_key_id => "**"
	secret_access_key => "***"
	region => "**"

}
}

filter {
	csv {
		separator => ","
		columns  => [ "name","uid","startdate","enddate"]

	}
	mutate {convert =>[uid , "integer"]}
	
	  date {
      match => [ "startdate", "ISO8601", "YYYY/MM/dd HH:mm:ss","YYYY/MM/dd HH:mm" ]
      target => "startdate"
      locale => "en"
	  timezone => "Asia/Dubai"
    }
	
	  date {
      match => [ "enddate", "ISO8601", "YYYY/MM/dd HH:mm","YYYY/MM/dd HH:mm:s" ]
      target => "enddate"
      locale => "en"
	  timezone => "Asia/Dubai"
    }
	  mutate {
    add_field => {
      "index_name" => "%{[@metadata][s3][key]}"
    }
   
  }

  }

output {
	elasticsearch {
		hosts => "localhost"
		index => "%{index_name}"
	}
	stdout{}

}

Any suggestion

Assuming that the S3 path is available in a field in the event you can use a grok filter to extract the name. How to extract a filename from a full path has been asked several times before so you should be able to find examples in the archives.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.