I have read about all XML filter import examples I can find. However, I just cannot figure out how to get this specific xml format imported through logstash.
Use case
The government of Denmark provides a public XML feed with data from the motorregistry. I would like to import this data into elasticsearch.
I wish to store it, to play with the analysis and types of data..
Anyway, the format contains nodes with danish names from the source (sorry).
XML Format
The repeating element for each registration is the node: ns:Statistik
I would like to just import the entire xml tree within ns:Statistik into a document type of the name "vehicle". The nodes should be converted into fields and the content with in, the value / nested object.
The XML file format can be found here:
Issue
Hardly any of my logstash.conf is working. I have gotten two different results when importing. I have succesfully imported the entire xml file into 1 document. I have succesfully imported each line into seperate documents.
But I don't understand the documentation of the XML filter apparently. CSV import etc. I know, and it seems so simply in comparison. The XML is really difficult to understand, and I cannot get any of the examples I've found on the forum nor stackoverflow to work either.
Hopefully one of you guys could provide me with a complete logstash conf, and from that I hopefully could learn how to do this with the XML filter in the future.
The split filter will separate the documents like you want to do. And then you have to rename and delete fields until you have a nice structure.
(If you have a configuration that is not working and are looking for help, it's always good to post what you've got. Solving a problem is easier, if you have something to build on. And one is probably more motivated, if the post is not basically saying 'Please do my job for me' )
I will take your feedback into account. It was in no way intended for a "Please do my job for me", I wish to learn how to work with this xml import and I have attempted for weeks now, also reaching out on IRC.
Nevertheless I appreciate your feedback and I have had a go on it. With your example it is clear to me first of all, how I misunderstood the XML filter documentation entirely. I attempted to make multiple nodes with xpath.
I still does not work entirely, but I would like to fiddle a little with your example and the split function. I am having the error _xmlparsefailure and _split_type_failure, but I think I can solve these.
Thank you for your example, it was just what I needed to move on
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.