Translate filter regex capture grouping

Hi, I am using Logstash to parse and enter graphite metrics into InfluxDB. However, I wanted to create a large dictionary source where it can match certain metric lines based on their pattern and reorganize the string based on the delimiter - "."

I know the translate filter can do this, also it can be done via mutate and gsub. I could not find documentation on how to use the regex to create capture groups and place them in the resultant string. Example:


I would like the above two to be reorganized to:

Notice that the fields are slightly swapped around, that's all. Would I be able to do this through some kind of capture group in translate (note the $1,2,3,4 are the captured groups)?

    field => "name"
    exact => true
    regex => true
    destination => "name"
    override => "true"
    dictionary => [
         "job\.(^[^.]+$)\.timer\.(^[^.]+$)\.processor\.(^[^.]+$)\.(.*)", "timer.$2.job.$1.processor.$3.$4"

If I need to use gsub for this, anyone know if I can source the mapping like translate can from a different file?

I do not believe the translate filter can do that. The code just clones the value of the dictionary entry.

mutate+gsub just processes the entries in the array in the configuration, it cannot read them from a file.

1 Like

That is what I figured, unfortunately. I suppose mutate+gsub would give me the best results for what I need, and I'm sure I can definitely similarly to the translate array of regex patterns but not from an external file, just a reaaaally long list in the actual filter. Thanks! I'll see if anyone has any secret/hidden input.

Note that the code for mutate+gsub is right there on github. You could repurpose it.

Use a ruby filter. In the init section read in a file and populate an array with fieldname / pattern / replacement triplets, like @gsub. Then run the section of code from the register function that generates @gsub_parsed.

In the code option run the code from the gsub function of a mutate filter.

Yeah that sounds like a good idea. Assuming my list of replacements isn't too long I will repurpose this and outsource the replacement hash to another file. Thanks again.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.