Best way to manage unbounded XML fields


(andrea) #1

Hi guys,
I'm wondering what's the best implementation of an index which stores XML data, written and processed by logstash, with unbounded elements.
Here is a simple example schema:

<xs:complexType name="root">
xs:sequence
<xs:element name="header_element" type="header_element_type" />
<xs:element name="body_element" type="body_element_type" maxOccurs="unbounded" />
</xs:sequence>
</xs:complexType>
<xs:complexType name="header_element_type">
xs:sequence
<xs:element name="trx_data" type="string" />
<xs:element name="name" type="string" />
</xs:sequence>
</xs:complexType>
<xs:complexType name="body_element_type">
xs:sequence
<xs:element name="general_data" type="string" />
<xs:element name="school_data" type="integer" />
</xs:sequence>
</xs:complexType> Blockquote

The first idea was using an index template as best practise, but then I was told that body_element can occurs many times :frowning:
What I would like to do is to avoid writting data of a single XML payload in multiple docs (n per n body + 1 per header and using an id field to relate them) because I suspect this will stress too much ES when doing data analytics.

Any idea or suggestion will be extremely appreciated.
Thank you guys!

Andrea


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.