Hi All,
The incoming feed (log) to Logstash has parameters which are delimited by ~|~
These are being mutated and split in Logstash as follows. :
if [type] == "tv_dmz_access" {
mutate {
split => ["message", "~|~"]
add_field =>{
"timeReqRecd" => "%{[message][0]}"
"remoteHostIP" => "%{[message][1]}"
"xForwardedFor" => "%{[message][2]}"
Problem is that "xForwardedFor"
at times receives more than one IPs which are comma separted. For example ~|~109.1.07.12, 14.1.15.4, 3.3.8.13, 4.15.24.7, 9.15.34.74~|~
Kibana tends to display only
the first IP and disregards other 4.
How can I set Logstash so that Kibana displays all 5 IPs instead of only one (first)?
Thanks
leandrojmp
(Leandro Pereira)
January 13, 2025, 2:39pm
2
Please share an example of what is being show into Kibana, it is not clear.
Also, share a complete example message so your pipeline, or at least, this part can be simulated to see what is the output.
Thanks.
The log being fed to Logstash is delimited by ~|~
and a snippet showing more than one IP addresses is as follows:
13/Jan/2025 00:00:01.328 -0500~|~29.60.36.110~|~-~|~16.25.81.76~|~79.51.52.21, 15.25.80.76, 29.132.216.290~|~GET~|~/web/fetchData?t=1736744400884~|~HTTP/1.1~|~200~|~329~|~1-04-TV-EPCVFSWFRR4JPJ36087429~|~www.xyz.com~|~-~|~TV-04~|~297866~|~-~|~TLSv1.2~|~ECDHE-RSA-AES128-GCM-SHA256~|~128
This is how it gets reflected in Kibana:
Please see the 3 comma separated IPs.
Also please note that the mapping has been corrected as follows so xForwardedFor is now the 5th field :
"timeReqRecd" => "%{[message][0]}"
"remoteHost" => "%{[message][1]}"
"X-Client-IP" => "%{[message][2]}"
"X-Akamai-True-Client-IP" => "%{[message][3]}"
"xForwardedFor" => "%{[message][4]}"
leandrojmp
(Leandro Pereira)
January 14, 2025, 3:37pm
4
So, what do you mean with:
Kibana tends to display only
the first IP and disregards other 4.
It is not clear what exactly you want, your screenshot is showing a string with all the ip addresses.
If you want this to be an array you just need to use the mutate split
filter.
filter {
mutate {
split => { "xForwardedFor" => "," }
}
}
This will create an array where each item is an ip addresses.
Thanks! Using mutate/ split was what I was looking for. Sorry for misleading information.