Filtering messages in Logstash

Hi All,

The incoming feed (log) to Logstash has parameters which are delimited by ~|~

These are being mutated and split in Logstash as follows. :

 if [type] == "tv_dmz_access"  {
                mutate {
                        split => ["message", "~|~"]
                        add_field =>{
                          "timeReqRecd" => "%{[message][0]}"
                          "remoteHostIP" => "%{[message][1]}"
                          "xForwardedFor" => "%{[message][2]}"

Problem is that "xForwardedFor" at times receives more than one IPs which are comma separted. For example ~|~109.1.07.12, 14.1.15.4, 3.3.8.13, 4.15.24.7, 9.15.34.74~|~

Kibana tends to display only the first IP and disregards other 4.

How can I set Logstash so that Kibana displays all 5 IPs instead of only one (first)?

Thanks

Please share an example of what is being show into Kibana, it is not clear.

Also, share a complete example message so your pipeline, or at least, this part can be simulated to see what is the output.

Thanks.

The log being fed to Logstash is delimited by ~|~ and a snippet showing more than one IP addresses is as follows:

13/Jan/2025 00:00:01.328 -0500~|~29.60.36.110~|~-~|~16.25.81.76~|~79.51.52.21, 15.25.80.76, 29.132.216.290~|~GET~|~/web/fetchData?t=1736744400884~|~HTTP/1.1~|~200~|~329~|~1-04-TV-EPCVFSWFRR4JPJ36087429~|~www.xyz.com~|~-~|~TV-04~|~297866~|~-~|~TLSv1.2~|~ECDHE-RSA-AES128-GCM-SHA256~|~128

This is how it gets reflected in Kibana:

Please see the 3 comma separated IPs.

Also please note that the mapping has been corrected as follows so xForwardedFor is now the 5th field :

                          "timeReqRecd" => "%{[message][0]}"
                          "remoteHost" => "%{[message][1]}"
                          "X-Client-IP" => "%{[message][2]}"
                          "X-Akamai-True-Client-IP" => "%{[message][3]}"
                          "xForwardedFor" => "%{[message][4]}"

So, what do you mean with:

Kibana tends to display only the first IP and disregards other 4.

It is not clear what exactly you want, your screenshot is showing a string with all the ip addresses.

If you want this to be an array you just need to use the mutate split filter.

    filter {
      mutate {
         split => { "xForwardedFor" => "," }
      }
    }

This will create an array where each item is an ip addresses.

Thanks! Using mutate/ split was what I was looking for. Sorry for misleading information.