Use file values as variable or iterate over values

[S-elk]

Hello!

Right now I have a pipeline that creates a field using a ruby scanner. When this scanner detects one of the values entered in the regular expression, it creates a field with that value.

For example:

if [x] {
    ruby {
      code => "event.set(y', event.get('[scanner]').scan(/\bvalue1\b|\bvalue2|\bvalueN/))"
}

is there any way to replace all this regular expression by a file or iterate over an array?

for example:

if [x] {
    ruby {
      code => "event.set(y', event.get('[scanner]').scan(/\b${iterate_over_values_of_array_in_file}\b/))"
}

Currently we already have more than 30 values and we will continue adding more and we would like this to be as dynamic as possible.

I don't know if the prune filter can solve this somehow.

any suggestions would be of great help.
Thank you

[Badger]

I think you would have to do it in ruby. You could read the regexp from a file when the pipeline starts up (i.e. in the init section of a ruby filter). You could event re-purpose code from the translate filter to periodically check if the file has been updated and reload it.

You could store the finished regexp in the file, or have one alternation per line and join them all up in the filter. It is just a question of how much ruby code you want to write.