Logstash parsing for dynamic fieldname

Elastic Stack Logstash
1

hi, I want to parse message:
My message part is looking like after applying json filter is
nodes.processes.C86BB2FAC5F22D51.user.name: value1
nodes.processes.C86BB2FAC5F22D51.user.sid: value2
nodes.files.EA68B2FAC5F22D51.behavioralCounters.BIEVASION: value3
node.files.EA68B2FAC5F22D51.behavioralCounters.BIGENERAL: value4

I want desired output is
user.name: value1
user.sid: value2
behavioralCounters.BIEVASION: value3
behavioralCounters.BIGENERAL: value4

I have tried filter with ruby code:

filter {
json{
source=> "message"
}
ruby {
code => '
# Get the "nodes" field value
nodes = event.get("nodes")

  # Check if "nodes" field is present and is a hash
  if nodes.is_a?(Hash)
    # Iterate through each key-value pair in "nodes"
    nodes.each_pair do |node_key, node_value|
      # Check if "processes" field is present and is a hash
      if node_value["processes"].is_a?(Hash)
        # Create a new hash to store the modified values
        new_node_value = {}

        # Iterate through each key-value pair in "processes"
        node_value["processes"].each_pair do |process_key, process_value|
          # Check if the dynamically generated key is present and is a hash
          if process_value.is_a?(Hash)
            # Copy the values from the dynamic key to the new hash
            process_value.each { |k, v| new_node_value[k] = v }
          end
        end

        # Remove the "processes" key from each "nodes" entry
        new_node_value.delete("processes")

        # Assign the modified values back to the original "node_value"
        node_value.replace(new_node_value)
      end

      # Remove the "nodes" key from each "nodes" entry
      node_value.delete("nodes")

      # Assign the modified "nodes" entry back to the event
      event.set("[node]", node_value)
    end

    # Remove the original "nodes" key
    event.remove("nodes")
  end
'

}
}

Result of the above code is ;
node.C86BB2FAC5F22D51.user.name: value1
node.C86BB2FAC5F22D51.user.sid: value2
node.EA68B2FAC5F22D51.behavioralCounters.BIEVASION: value3
node.EA68B2FAC5F22D51.behavioralCounters.BIGENERAL: value4

can anyone do it for my desired output ,since my code does not remove dynamic field name (parent3). It just delete nodes.processes and nodes.files and replace with node. I want to remove third dyanmic field also.

2

When I run your ruby filter against

     "nodes" => {
        "files" => {
        "EA68B2FAC5F22D51" => {
            "behavioralCounters" => {
                "BIEVASION" => "value3",
                "BIGENERAL" => "value4"
            }
        }
    },
    "processes" => {
        "C86BB2FAC5F22D51" => {
            "user" => {
                 "sid" => "value2",
                "name" => "value1"
            }
        }
    }

I get

      "node" => {
    "C86BB2FAC5F22D51" => {
        "user" => {
             "sid" => "value2",
            "name" => "value1"
        }
    }
},

which is not what you say you are getting. Not sure how to explain that.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.