Logstash - Trying to assign an existing key to a new key's value

Elastic Stack Logstash
1

Hello,
I’m pulling aggregated events from Opentsdb and I have a key/value where the key is the epoch event time and value is the agg count.
Example:

"dps" => {
    "1594657920" => 12.0
},

I’m trying to find a way to break this key/value into two key/value fields, “epoch”: "1594657920" and “count”: 12.0. I’ve been trying but not successful. I’m new to Logstash and Ruby
Any assistance would be appreciated!

Here’s my config:

input {
  http_poller {
    urls => {
      OpenTSDB => {
        # Supports all options supported by ruby's Manticore HTTP client
        method => get
        url => "http://some.very.long.url"
        headers => {
          Accept => "application/json"
        }
     }
    }
    request_timeout => 60
    # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
    schedule => { cron => "* * * * * UTC"}
    codec => "json"
    # A hash of request metadata info (timing, response headers, etc.) will be sent here
    metadata_target => "http_poller_metadata"
  }
}
filter {
  ruby {
    code => "
      wanted_fields = ['@timestamp','dps','tags']
        event.to_hash.keys.each { |k|
        event.remove(k) unless wanted_fields.include? k
      }
        event.get('[dps]').each { |key, value|
        event.set('[' + key + ']', value)
      }
    "
  }
  if [tags][response_code] {
    mutate {
      add_field => {
        "response_code" => "%{[tags][response_code]}"
        "SnRC" => "%{[tags][response_code]}/%{[tags][error_reason_code]}"
      }
    }
  }
}
output {
  stdout {
    codec => rubydebug
  }
}

Here’s my output so far:

{
       "@timestamp" => 2020-07-13T16:33:01.105Z,
             "tags" => {
                      "api" => "GET-/what/you/want",
            "response_code" => "200",
            "business_flow" => "NULL",
               "partner_id" => "NULL",
        "error_reason_code" => "NULL",
          "app_instance_id" => "0",
                "component" => "some-api",
                     "host" => "someservername"
    },
    "response_code" => "200",
             "SnRC" => "200/NULL",
              "dps" => {
        "1594657920" => 12.0
    },
}

Thanks in advance

2

I ask pretty much same question and got answer here

https://discuss.elastic.co/t/flatten-json-and-create-dynamic-field/240360/9

3

but for you case it is even simple

if [dps] {
   ruby {
     code => '
         event.get("[dps]").each { |k,v|
           event.set("epoch", k)
           event.set("count", v)
         }
         event.remove("dps")
   '
   }
}

this code says if you have something in field dps
go throucsh eash key,value pair
and set epoch = key and count=value
and remove field dps

4

That worked great, thank you!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.