Translate filter regex capture grouping

Elastic Stack Logstash
1

Hi, I am using Logstash to parse and enter graphite metrics into InfluxDB. However, I wanted to create a large dictionary source where it can match certain metric lines based on their pattern and reorganize the string based on the delimiter - "."

I know the translate filter can do this, also it can be done via mutate and gsub. I could not find documentation on how to use the regex to create capture groups and place them in the resultant string. Example:

job.cm_attributes.timer.CMTopology.processor.DiscoveryResponse.succeeded
job.cm_equalization.timer.CMHierarchy.processor.ExplorerResponse.succeeded

I would like the above two to be reorganized to:
timer.CMTopology.job.cm_attributes.processor.DiscoveryResponse.succeeded
timer.CMHierarchy.job.cm_equalization.processor.ExplorerResponse.succeeded

Notice that the fields are slightly swapped around, that's all. Would I be able to do this through some kind of capture group in translate (note the $1,2,3,4 are the captured groups)?

    field => "name"
    exact => true
    regex => true
    destination => "name"
    override => "true"
    dictionary => [
         "job\.(^[^.]+$)\.timer\.(^[^.]+$)\.processor\.(^[^.]+$)\.(.*)", "timer.$2.job.$1.processor.$3.$4"
        ]
  }

If I need to use gsub for this, anyone know if I can source the mapping like translate can from a different file?

2

I do not believe the translate filter can do that. The code just clones the value of the dictionary entry.

mutate+gsub just processes the entries in the array in the configuration, it cannot read them from a file.

3

That is what I figured, unfortunately. I suppose mutate+gsub would give me the best results for what I need, and I'm sure I can definitely similarly to the translate array of regex patterns but not from an external file, just a reaaaally long list in the actual filter. Thanks! I'll see if anyone has any secret/hidden input.

4

Note that the code for mutate+gsub is right there on github. You could repurpose it.

Use a ruby filter. In the init section read in a file and populate an array with fieldname / pattern / replacement triplets, like @gsub. Then run the section of code from the register function that generates @gsub_parsed.

In the code option run the code from the gsub function of a mutate filter.

5

Yeah that sounds like a good idea. Assuming my list of replacements isn't too long I will repurpose this and outsource the replacement hash to another file. Thanks again.