Hello,
My colleague @Anabella_Cristaldi and me have an XML like the following structure
<animal>
<h>feline</h>
<h>meow</h>
<p>
<property>
<character>
<breed>
<age>
</property>
<cat name="gardfield">
<c>good</c>
<c>persian</c>
<c>1year</c>
</cat>
<cat name="felix">
<c>regular</c>
<c>common</c>
<c>3year</c>
</cat>
<cat name="misifu">
<c>sweet</c>
<c>common</c>
<c>1year</c>
</cat>
</p>
<p>
<property>
<height>
</property>
<cat name="gardfield">
<c>3</c>
</cat>
<cat name="felix">
<c>8</c>
</cat>
<cat name="misifu">
<c>2</c>
</cat>
</p>
</animal>
and we have an similar XML for dogs
We need to have in ES a document by animal with all the properties.
We managed to do it using multine codec in the input plugin, xml and split filter plugin and in the output plugin action => "update" and doc_as_upsert => true and fix id for every cat/dog.
Originally our pipeline.workers was set up to 3 but we have noticed that sometimes we had erratic results. In a cat document we had dog properties.
Then we set the pipeline.workers to 1 and we have the expected results.
How pipeline.workers works and why we should use only one?
Thanks a lot!
PS: we use cats and dogs for simplicity, but we have data from network equipments in XML format