Remove part of field name in Logstash

I am using Logstash 6.0.0

I am receiving metrics with different application Ids. How can I delete the application Id in the field name?

application_1512057801087_0235.driver.BlockManager.memory.maxMem_MB => driver.BlockManager.memory.maxMem_MB

A quick solution would be some custom Ruby code, especially if you don't know the application id patterns beforehand, like

ruby {
    code => "
        event.set('fieldname', event.get('fieldname').split('.')[1..-1].join('.'))

It's kinda overkill, but it works (unless there are dots in the application id itself).

Ho w can I indicate that the ruby code applies only on field starting with "application_..."

filter {
    if [type] == "spark" {
         ruby {
                code => "event.set('fieldname', event.get('fieldname').split('.')[1..-1].join('.')"

With a dynamic field name you'll have to iterate over all fields.

if event.to_hash.each { |k, v|
  if k.start_with? 'application_'
    event.set('application', v)
1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.