I am trying to create a geo_point field to add coords using the http poller to pull some data in json format.
so the json is coming in a format like this. Multiple JSON documents representing each item and it's properties and sub-properties
[
{
"1": "a",
"2": "",
"3": "",
"4": {
"4a": {
"4a1": ""
},
"4b": {
"4b1": ""
},
"4c": {
"latitude": 01,
"longitude": -01
}
}
},
{
"1": "b",
"2": "",
"3": "",
"4": {
"4a": {
"4a1": ""
},
"4b": {
"4b1": ""
},
"4c": {
"latitude": 01,
"longitude": -01
}
}
}
]
so for my config i have
input {
http_poller {
urls => {
url => {
method => get
url => ""
headers => {
Accept => "application/json"
}
}
}
request_timeout => 60
interval => 60
codec => "json"
metadata_target => "http_poller_metadata"
}
}
filter {
mutate {
add_field => { "[location][lat]" => "%{[4][4c][latitude]}" }
add_field => { "[location][lon]" => "%{[4][4c][longitude]}" }
}
mutate {
convert => { "[location]" => "geo_point" }
}
mutate {
convert => { "[location][lat]" => "float" }
convert => { "[location][lon]" => "float" }
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
}
stdout { codec => rubydebug }
}
The http poller pulls the data in wonderfully without the mutate. i can also mutate and add a field such as test_field: test_data without issue. anything I start using anything regarding nested data it falls on its face. So, I'm trying to figure out where my formatting is off for the filters. is [4] the true top-level field or is it the item in the array? or do i need to split them first?