Using dissect dynamicly on filepath

[ThePreMan]

Hi. Im trying to setup a workflow where people can upload there log files onto a webserver and while doing so add tags to the log file for further differenciation. I was thinking of putting tags into the file name like filename-tag1-tag2-tag3-tag4-tagN.log. Then logstash or filebeat should be able to run a dissect operation which then would lead to the tags being shown in kibana under tags.
First: Do you think that is possible ? And second: can you build it to be dynamic no matter how many tags you put in ?

[Badger]

You could make a copy of the filename using mutate+add_field, then use mutate+gsub to remove filename- and .log, then use mutate+split to divide the list of tags into an array. If there is a possibility that the [tags] field already exists then I think you would need to use a ruby filter to merge the two arrays.

[ThePreMan]

That sounds good. That way I could also put the filename without the path into an extra field.
Thank you I will try that

[ThePreMan]

How do you remove things wich gsub. In the gsub section within the mutate doc it only states it to replace certain characters .
What I did do:

 mutate {
   convert => {
     "PTimestamp" => "integer"
     "Count" => "integer"
     "Index" => "integer"
     "#Args" => "integer"
     "SessionID" => "integer"
   }
   copy => {
     "log.file.path" => "NameTags"
   }
   gsub {
     "Filename", "/", " "

   }
   split => { "NameTags" => "," }
   add_tag => [ "foo_%{NameTags}" ]
 }

[Badger]

@ThePreMan

How do you remove things wich gsub.

If you have a field that contains 'filename-tag1-tag2-tag3-tag4-tagN.log' then you can remove parts of it using

mutate { gsub => [ "someField", "filename-", "", "someField", "\.log", "" ] }