Dynamically move nested fields to the top level

Elastic Stack Logstash
1

Hello,

I am trying to parse an XML message with lots of nested fields. I am using the XML filter to parse the data and send it to Elasticsearch. The issue that I am having is that the XML path to the value is too large.

For example, this is what is being shown in the Kibana discover: These are nested array fields

parsed.Invoices.Invoice.Header.InvoiceId
parsed.Invoices.Invoice.Receiver.CustomerInformation.CustomerName
parsed.Invoices.Invoice.Receiver.CustomerInformation.CustomerID
parsed.Invoices.Invoice.Receiver.CustomerInformation.Address.StreetAddress1
parsed.Invoices.Invoice.Rows.Row.RowNumber
parsed.Invoices.Invoice.Rows.Row.Product.ProductId

Is there a way to use the ruby filter to move these fields up such as below without explicitly specifying each path in the ruby code?

I would like it to be like:

Header.InvoiceId
CustomerInformation.CustomerName
CustomerInformation.CustomerID
Address.StreetAddress1
Row.RowNumber
Product.ProductId

This is what I have for the ruby code:

ruby {
    code => '
        event.get("parsed").each { |k, v|
            event.set(k,v)
        }
        event.remove("parsed")
    '
}

This will move everything just one level up such as below but still lost on how to move the rest of the fields up more levels dynamically.

   Invoices.Invoice.Header.InvoiceId
   Invoices.Invoice.Receiver.CustomerInformation.CustomerName
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.