Creating a dynamic S3 tag

Elastic Stack Logstash
1

I have log input coming from multiple sources that I am writing into S3 buckets but now I am wanting to add the source name (machine name) to the S3 filename as a tag so my requirement is a dynamic tag name based on the different machine names that will then be added as a tag.

My current configuration is-

filter {
	mutate {
		add_field => {
			"filepath" => '%{[log][file][path]}' 
		}
	}
	
	grok {
		match => ["filepath", "%{WORD:filepath}.log"]
		overwrite => ["filepath"]
	}
 }

output {
     s3 {
          ....
	 tags => ["%{filepath}"]
    }
}

The filenames in S3 is

ls.s3.3791f296-2b0e-4750-a868-a668a5b98e61.2020-03-20T12.32.tag_%{filepath}.part8

So what am I missing and can what I want to do be done with the S3 tag property.

TIA,
Bill

2

No, it cannot be done. The code that adds tags is just a concatenation of strings in an array. It does not event.sprintf each tag. There has been an issue open for this for a couple of years.

3

Thanks for the quick response Badger and I will look at another approach to this probably something like if [field][machinename] = xxxx s3 tag => 'machinename'

I'll let me team know this.

Bill

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.