How to parse array[] in json

[spc1jh (jihoon choi)]

Hi all,

I want to monitir aws waf2 logs with elastic search.

I tried to insert data in elasticsearch with logstash.
I faced some problems.

shortly below is the json format data

{
  "nonTerminatingMatchingRules" : [
	{
	  "ruleId" : "AWS-AWSManagedRulesCommonRuleSet",
	  "action" : "COUNT"
	}
  ],
  "ruleGroupList" : [
	{
	  "excludedRules" : null,
	  "nonTerminatingMatchingRules" : [ ],
	  "ruleGroupId" : "arn:aws:wafv2:xxxx:xxxx:regional/rulegroup/HTTP_Header_Injection/xxxxx",
	  "terminatingRule" : null
	},
	{
	  "excludedRules" : null,
	  "nonTerminatingMatchingRules" : [ ],
	  "ruleGroupId" : "AWS#AWSManagedRulesCommonRuleSet",
	  "terminatingRule" : {
		"ruleId" : "SizeRestrictions_BODY",
		"action" : "BLOCK"
	  }
	}
  ],         
  "action" : "ALLOW",
}

I really want to parsing ruleGroupList area.

If I use split filter in logstash, only one data is remained as below.

filter {
  split {   field => "[ruleGroupList]"    }
}

==>

ruleGroupList" : {
            "ruleGroupId" : "AWS#AWSManagedRulesCommonRuleSet",
            "nonTerminatingMatchingRules" : [ ],
            "terminatingRule" : {
              "action" : "BLOCK",
              "ruleId" : "SizeRestrictions_BODY"
            },
            "excludedRules" : null
          },

only one data is remaind, even if two data was exist originally.
How to parse correctly, please let me know.

Thanks.

[Rahul_Kumar4 (Rahul Kumar)]

@spc1jh

Try using the json filter before passing it through the split filter.

filter {
  
  json{
    source => "message"
  }

  split {   field => "[ruleGroupList]"    }
}

Also your raw log has that trailing , after the action field which makes it an invalid JSON, so you may want to fix that before giving this a try.

[spc1jh (jihoon choi)]

Thanks,
but I already used json filter.
here is the filter rules that I am using.

    json  {
       source => "message"
     }

     geoip {
       source => "[httpRequest][clientIp]"
       target => "geoip"
       add_tag => [ "waf-geoip" ]
     }

     split {
       field => "[ruleGroupList]"
     }

     date {
         match => ["timestamp", "UNIX_MS"]
         target => "timestamp"
     }