Continuing the discussion from I would like to access nested data from an array:
The data inside the array are not getting indexed and hence not displayed on the kibana dashboard. What is the best way to process the data from Logstash > elastic search > kibana. I wanted to get the data in the results and get them displayed in kibana. Sample of my data below. Presently the data is displayed as single object on kibana (discover)
Any suggestions would be very much appreciated.
{
"meta":{
"paginationTotalResults": 3,
"response_in_sec": 0.08
},
"results":[
{
"pageTitle": "Breast cancer | Cancer Research UK",
"url": "http://www.cancerresearchuk.org/about-cancer/type/breast-cancer/",
"browserUri": "browsers/ff",
"dataStartDuration": 0.067,
"renderStart": 3.071
}
{
{
"pageTitle": "Breast cancer | Cancer Research UK",
"url": "http://www.cancerresearchuk.org/about-cancer/type/breast-cancer/",
"browserUri": "browsers/ff",
"dataStartDuration": 0.094,
"renderStart": 2.266
}
]
}
Config file used in logstash
input {
http_poller{
urls => {
test1 => {
method => get
url => "https://xxxxxxxx.com/pa/1/testRuns?jobTemplate=jobTemplates/76045"
headers => {
Authorization => "Bearer c59e764ccad6ffc0a5ca28eb50255ae910b126190baa5e020e9da5403660"
Realm => "626367"
}
}
}
request_timeout => 1200
interval => 1200
codec => json
# A hash of request metadata info (timing, response headers, etc.) will be sent here
#metadata_target => "http_poller_metadata"
}
}
filter{
}
output {
#stdout { codec => ruby_debug }
elasticsearch {
hosts => "http://localhost:9200"
index => "api-index"
}
}