How to parse nested json events

Elastic Stack Logstash
1

Hello!
I have nested json objects like this :

{
  "_index": "impe_logs-2021.11.19-1",
  "_type": "_doc",
  "_id": "tyu",
  "_version": 1,
  "_score": null,
  "_source": {
    "port": 54640,
    "impe": {
      "abp": {
        "random_id": "",
        "policy_id": "91",
        "token_id": "",
        "bot_violations": [
          "known_violator_header_id",
          "known_violator_user_agent",
          "no_token",
          "suspicious_user_agent"a
        ]}
"client": {
      "geo": {
        "name": ""
      },
      "domain": "",
      "ip": "1.2.3.4"
    }
	}}}

when im trying to store in Elasticsearch :
impe.abp.policy_id 91 but i want it to be stored as policy_id 91
and client.geo.name want it as client geo name

2

You can use the mutate filter to copy the contents of one field into another (under a different name) or rename the field all together.

For example:

filter {
   mutate {
      rename => { "[impe][abp][policy_id]" => "policy_id" 
                             "[client][geo][name]" => "client_geo_name"}
   
}

However, I would caution that you should try and conform to ECS as much as possible :slight_smile:

1 Like
3

Hello @AquaX ,
Thanks for the help ,yes even mutate works.
but found a solution from other discussion:

ruby {
         code => '
                event.get("[imperva][abp]").each { |k, v|
                event.set(k,v)
                }
         event.remove("[imperva][abp]")
}
4

Ahh! I did not understand that you wanted to do this programmatically for a dynamic list. Glad you got it figured out!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.