Gsub specific characters but not all of them in value

Elastic Stack Logstash
1

Hi,

I am desperately trying to figure out a way to replace full stops in an alphabetic string in a value without affecting decimal values. I am faced with two problems, I have a field value with an unknown quantity of full stop delimited words and the second problem is that "gsub" replaces all occurrences.

Here is an example of my data:

payload="DATA1-DATA1.DATA2.DATA3=10.10,DATA1-DATA1.DATA2.DATA4=20.20,DATA1-DATA1.DATA2.DATA5=30.30"

or

payload="DATA1-DATA1.DATA2.DATA3.DATA4=10.10,DATA1-DATA1.DATA2.DATA3.DATA5=20.20,DATA1-DATA1.DATA2.DATA3.DATA6=30.30"

Both of the above will be passed to KV to create new fields but using the following gsub will replace the dots in the decimal values which I don't want:

mutate { gsub => [ "payload", "[.]", "-" ] }

Ultimately I want to use kv to create the following from both payloads:

DATA1-DATA1-DATA2-DATA3=10.10
DATA1-DATA1-DATA2-DATA4=20.20
DATA1-DATA1-DATA2-DATA5=30.30

DATA1-DATA1-DATA2-DATA3-DATA4=10.10
DATA1-DATA1-DATA2-DATA3-DATA5=20.20
DATA1-DATA1-DATA2-DATA3-DATA6=30.30

Does anyone know if ruby perhaps can be used to replace only the full stops with hyphens if surrounded by words, or perhaps replace the full stops if surrounded by numbers to a completely different character so that I can use gsub on the whole payload to replace the full stops and then use kv?

2

You are looking for (zero-width) lookahead/lookbehind assertions. A pattern like

(?<=[A-Z])\.(?=[A-Z])

should only match full stops that are preceded by a letter and followed by a letter.

3

Thank you Magnus, that is perfect!

I dont suppose you know a way of converting all fields starting with a specific string to float do you?

Something like:

mutate { convert => { "metric-*(.+)" => "float" } }

4

You'll have to use a ruby filter for that. Something similar to

ruby {
  code => "
    event.to_hash.each { |k, v|
      event[k] = v.to_f if k.start_with? 'metric-'
    }
  "
}

should work.