How to split and label fields of an array of arrany?

Chuan_Ydnk · November 12, 2018, 6:20am

Hi

I have the following output after i had used filter { json { source => "message" } }

Now, I wanted to split the array of objects

The first field is link_name, link_alias, bto,bcs,bss.

I am sorry really a newbie in logstash

"formattedData":[["10.212.75.73 WAN optimized","10.212.75.73 WAN optimized","5.0254744E7","2.70668E7","2.3187946E7"],["10.212.75.73 WAN pass through","10.212.75.73 WAN pass through","4.5109216E7","1.1213042E7","3.3896176E7"],["172.18.66.5 wan0_0","BJS11ACC001-WAN0_0","9149226.0","1973337.125","7175889.0"],["57.28.59.5 wan0_0","BOM11ACC001-WAN0_0","6082061.0","3228388.0","2853673.0"],["57.28.84.5 wan0_0","BOM41ACC001-WAN0_0","5317941.5","3211377.25","2106564.25"],["172.18.80.5 wan0_0","CAN41ACC001-WAN0_0","1.6712449E7","1.0740312E7","5972137.0"],["172.28.11.82 wan2_0","HKGX3ACC0001-WAN2_0","9.0989192E7","4.2567948E7","4.842124E7"],["172.28.11.82 wan3_0","HKGX3ACC0001-WAN3_0","3.3666396E7","1.3425408E7","2.024099E7"],["10.164.6.28 GigabitEthernet0/0/1","PMUC2345-WAN0_0-GE0_0_1","1.9228112E7","2586036.75","1.6642074E7"],["57.28.18.22 wan0_0","TPE11ACC001-WAN0_0","8087264.5","1268966.75","6818298.0"]]

Chuan_Ydnk · November 12, 2018, 8:15am

Hi there,

i need some help here, i manage to use spit and mutate to achieve "nearly" what i wanted but however, the [formattedData][1] dosen't return me the correct value. What is it so ?

input {
http_poller {
urls => {
test1 => {
url => "https://wecacascxv01.assurance.sita.aero/rest/dmiquery/getDMIData3"
method => post
user => "xxxxx"
password => "xxxxx"
headers => {
"Content-Type" => "application/json"
}
body => '{"appId": "CVENT","viewId": "TfcOnLink","dataSourceId": "ALL_AGGR","dimensionIds": ["msrPoint","msrPoint4Cust"],"metricIds": ["BTO","BCS","BSC"],"dimFilters": [["msrPoint4Cust","WAN",false]],"metricFilters": [["BTO",">",5000000,1]],"sort": ,"top": 1000,"resolution": "r","timePeriod": "p","numberOfPeriods": 1}'
}
}
request_timeout => 60
schedule => { "every" => "60s" }
# A hash of request metadata info (timing, response headers, etc.) will be sent here
# metadata_target => "http_poller_metadata"
tags => dcrum_endpoints
}
}

filter {
json {
source => "message"
}
split {
field => "[formattedData]"
}
split {
field => "[formattedData][0]"
}
mutate {
remove_field => ["rawData","dimensionIds","metricIds","columnHeader","columnUnit","columnRendererIds","response_message","columnHeaderName"]
}
mutate {
rename => { "[formattedData][0]" => "Link name" }
rename => { "[formattedData][1]" => "Link alias" }
rename => { "[formattedData][2]" => "Total bandwidth usage" }
rename => { "[formattedData][3]" => "Incoming bandwidth usage" }
rename => { "[formattedData][4]" => "Outgoing bandwidth usage" }
rename => { "[formattedData][5]" => "Outgoing bandwidth usage5" }
rename => { "[formattedData][6]" => "Outgoing bandwidth usage6" }
rename => { "[formattedData][7]" => "Outgoing bandwidth usage7" }
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
index => "wecascxsv01-json-%{+YYYY.MM.dd}"
}
file {
path => "/tmp/debug-%{+YYYY.MM.dd}.txt"
}
stdout {
codec => rubydebug
}

Here's the output:

I don't understand why [formattedData][1] should be return the BJS11ACC001-WAN0_) but it returned the Total bandwidth attribute which is the third attribute of the formattedData ? anyone can explain ?

Chuan_Ydnk · November 12, 2018, 8:22am

Any kind souls able to help ?

system · December 10, 2018, 8:22am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Logstash parse object Logstash	5	653	January 20, 2021
Split Json Array Objects Logstash	9	1780	April 15, 2020
Logstash filter to split field into fields of arrays Logstash	5	428	May 2, 2018
Split array with different nested elemetnts Logstash	7	1713	March 14, 2022
How can I parse array of objects using Logstash? Logstash	7	4991	May 19, 2020

How to split and label fields of an array of arrany?

Related topics