Converting strings to integers then manipulating

[sdberts]

I have a string coming through in the following format:

"Memory Used": "500 MB"
"Memory Used": "768 KB"

Basically, what I need to do is convert everything into MBs in integer/float format. For the fields containing "MB," it's simple enough. I just do a gsub and replace "MB" with "". I'm having a bit of trouble with the fields containing "KB" though. What I need to do is strip off KB if it exists, then divide the new integer by 1000 to get MBs. Any advice?

[Jenni]

I'm on my phone. So I won't even try to write working code. But what I'd do is write a ruby filter:

Initialize a variable divisor = 1
If the field value matches .*kb$ divisor = 1000
Replace everything that is not a number [^\d] with nothing (gsub)
Convert the string: Integer()
Divide this by the divisor.

I hope, that helps.

[magnusbaeck (Magnus Bäck)]

Have you looked at the (third-party) units filter?

[Badger]

If you don't care about the difference between 1000 and 1024 then the following might work for you. I use it when parsing GC logs

  mutate {
    gsub => [ "message", "\b(?<size>[0-9]+)\.[0-9](?<unit>[BKMG])\b", "\k<size>\k<unit>" ]
    gsub => [
      "message", "\b(?<size>[0-9]+)G\b", "\k<size>000000000",
      "message", "\b(?<size>[0-9]+)M\b", "\k<size>000000",
      "message", "\b(?<size>[0-9]+)K\b", "\k<size>000",
      "message", "\b(?<size>[0-9]+)B\b", "\k<size>"
      ]
   }
ruby { code => 'event.set("someFieldInMB", event.get("someField").to_f/1000000)' }

[sdberts]

I have not, but I'll check that out!

[Jenni]

The filter plugin will probably solve this, but for the sake of completeness, this is what I had in mind:

ruby {
    code => '
      divisor = 1
      divisor = 1000 if event.get("Memory Used") =~ /KB$/
      event.set("Memory Used", Integer(event.get("Memory Used").gsub(/[^\d]/,"")).to_f/divisor)
    '
  }

[sdberts]

This is almost exactly what I ended up with based on your previous comment. Thank you very much!