XML filter

Elastic Stack Logstash
#1

Hello,

I have to parse some xml files (with xml filter) and store each child item (car) as a document in Elasticsearch but I don't see how I can do that.

My xml file could containt n child items (around 100-200 per xml file) and I need 1 record for each child. There is an example :

<cars> <car> <model>BMW</model> <color>Red</color> </car> <car> <model>Audi</model> <color>Yellow</color> </car> <car> <model>Mercedes</model> <color>Black</color> </car> </cars>

So I want to have 1 record for each car contained in my xml document. So the xpath param will be something like that
xpath => [ "/cars/car/model", "Model", "cars/car/color", "Color"]
But I don't get the logic of how can I tell Logstash to store each item in a new record =/

Could you explain me please?

#2

You should be able to use the split filter as long as you can convert the XML into something like this:

{
  "cars": {
    [
      {
        "model": "BMW",
        "color": "Red"
      },
      {
        "model": "Audi",
        "color": "Yellow"
      },
      ...
    ]
  }
}

I wonder, is the xpath option really the best way forward? Can't you parse the whole XML document, delete any unwanted fields, and use the split filter on the result?

#3

Yop @magnusbaeck thanks for your response.

Finally I just used the following filter

xml { source => "cars" target => "cars" }

So in elastic I have one document for each car with following message field and it's exatcly what I want.
<car> <model>xxx</model> <color>yyy</color> </car>
Once parsed like this, I can easly add fields model or color to have a more clean document (instead of a message field with all data inside).

:ok_hand:

#4