Pulling Nested XML Data from a http API for loading into Elasticsearch

Elastic Stack Logstash
1

Ok first time poster - so please go easy on me - two questions.

I would like to be able to pull data with logstash from an API. The API is at an address that looks like the following;
http://domain.com/reports/1000000
http://domain.com/reports/1000001
http://domain.com/reports/1000002
....
http://domain.com/reports/9999999

Is there are way to make incremental http requests to pull this data into logstash so it can be pushed into elasticsearch? Or is there a recommended tool for this purpose?

Here is the data that will be returned by the API.

`

123456 2015-01-01 COKE FANTA `

As I am sure you know logstash will convert this to the following JSON

For the first name.
"name" => [
[0] "COKE"
]

For the second name
"name" => [
[0] {
"type" => "M",
"content" => "FANTA"
}
]

And when I try to push this into elastic it will fail because the mapping will expect a string for the name field or expect an object - it cant't have both. How best to deal with this problem. I want to keep the party type and name details together - so xpathing just the name contents out does not meet my requirements.

2

Thanks Guys I am over the moon able the latest logstash release which included the force_content configuration. Basically the below configuration

filter {
xml {source => "message"
force_content => "true"
target => "parsed"}
}

Gives the below output which is now consistent even when some fields have attributes while others fields have not attributes.

"parsed" => {
 "xmlns" => "http://domain.com/schema/reporting",
"header" => [
    [0] {
        "repNumber" => [
            [0] {
                "content" => "123456"
            }
        ],
          "repDate" => [
            [0] {
                "content" => "2015-01-01"
            }
        ]
    }
],
 "party" => [
    [0] {
        "type" => "R",
        "name" => [
            [0] {
                "content" => "COKE"
            }
        ]
    },
    [1] {
        "type" => "S",
        "name" => [
            [0] {
                   "type" => "M",
                "content" => "FANTA"
            }
        ]
    }
]

}
}