Is there a for loop to iterate from %{header1} to %{header40}

Hello,

I was wondering if there exists a more elegant way to do this.
Currently, in my pipeline to process .log files, I used grok filter to filter field header and field data.
This allows me to achieve 40 fields & 40 fields data.

To properly display it on kibana, i did this:

if [header40] =~ /.+/ {
	mutate {
      add_field => { "%{header1}" => "%{header1_data}" }
      remove_field => [ "header1", "header1_data" ]
      .......  (repeats for 40 times)
      .......
      add_field => { "%{header40}" => "%{header40_data}" }
      remove_field => [ "header40", "header40_data" ]
      }
 }

Is there a way to iterate this using a for loop so that it can be short & concise?

    ruby {
        code => '
            for i in 1..40
                k = "header#{i}"
                s = event.get("#{k}_data")
                if s
                    event.set(k, s)
                    event.remove("#{k}_data")
                end
            end
        '
    }

It is ugly, but it works.

Thank you! It works but the header field remains as header_(n)
I refine the code alittle and this was the outcome:

 ruby {
      code => '
        for i in 1..40
          k = "header#{i}"
          s = event.get("#{k}_data")
          j = event.get("#{k}")
          if s
            event.set(j, s)
          else
            event.set(j, "")
          end
          event.remove("#{k}_data")
          event.remove("#{k}")
        end
      '
  }

Nevertheless, thank you for this. It helped

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.