XML: converting each tag attribute as the actual keys

[thdesy (Thomas Hartmann)]

Hi,

I have a somewhat XML log, where the attributes are the actual keys, e.g., [1]

each event is in a <c> tag and every message key/value is in a <a> tag. The actual key name is the n attribute and the value is in a 'file type' tag, so <i> for integers, <s> for strings and <r> for floats.

So, I attempted to select all /c/a/title() or text() via xpath and 'somehow' use the result to set the destination field with something like

input {
  file {
    path => "/var/log/loggy.xml"
    start_position => "beginning"
    exclude => "*.gz"
    type => "xml"
      codec => multiline {
	pattern => "<c>"
        negate => "true"
        what => "previous"
      }
  }
}

filter{
    xml{
        source => "message"
	store_xml => true
        target => "events"
        namespaces => {
          "xsl" => "http://www.w3.org/1999/XSL/Transform"
          "xhtml" => "http://www.w3.org/1999/xhtml"
        }
	xpath => ["/c/a/text()","text()"]
    }
}

However, I have not managed to select all the attribute elements and convert them to destiantion fields

Maybe somebody has a tip for me, how to transform the events properly?

Later on, I probably would try to either mutate:gsub the file type tags to empty strings (don't know if mutate:convert could be used from a condition)

Cheers and thanks for ideas,
Thomas

============================

[1]

<c>
    <a n="Proc"><i>0</i></a>
    <a n="Cluster"><i>64866</i></a>
    <a n="EventTime"><s>2020-09-17T16:40:17.878</s></a>
    <a n="MyType"><s>ExecuteEvent</s></a>
    <a n="ExecuteHost"><s>79655.0</s></a>
    <a n="Subproc"><i>0</i></a>
    <a n="EventTypeNumber"><i>1</i></a>
</c>
<c>
    <a n="SentBytes"><r>0.0</r></a>
    <a n="TotalRemoteUsage"><s>Usr 1 05:24:22, Sys 0 01:47:25</s></a>
    <a n="TotalLocalUsage"><s>Usr 0 00:00:00, Sys 0 00:00:00</s></a>
    <a n="EventTypeNumber"><i>5</i></a>
    <a n="TotalSentBytes"><r>0.0</r></a>
    <a n="Subproc"><i>0</i></a>
    <a n="MyType"><s>JobTerminatedEvent</s></a>
    <a n="RunRemoteUsage"><s>Usr 1 05:24:22, Sys 0 01:47:25</s></a>
    <a n="EventTime"><s>2020-09-17T16:40:37.992</s></a>
    <a n="Cluster"><i>62772</i></a>
    <a n="Proc"><i>0</i></a>
    <a n="ReceivedBytes"><r>0.0</r></a>
    <a n="TerminatedNormally"><b v="t"/></a>
    <a n="TotalReceivedBytes"><r>0.0</r></a>
    <a n="ReturnValue"><i>0</i></a>
    <a n="RunLocalUsage"><s>Usr 0 00:00:00, Sys 0 00:00:00</s></a>
</c>

[Badger]

It is not entirely clear what you want to do, but I will take a guess You will not be able to do it with xpath as far as I know, you would have to use ruby...

    xml {
        source => "message"
        target => "events"
        force_array => false
        store_xml => true
        namespaces => {
            "xsl" => "http://www.w3.org/1999/XSL/Transform"
            "xhtml" => "http://www.w3.org/1999/xhtml"
        }
    }
    ruby {
        code => '
            e = event.get("events")
            if e.is_a? Hash
                e["a"].each { |x|
                    key = x["n"]
                    if x["s"]
                        value = x["s"]
                    elsif x["i"]
                        value = x["i"].to_i
                    elsif x["r"]
                        value = x["r"].to_f
                    elsif x["b"]
                        value = (x["b"]["v"] == "t")
                    end
                    event.set(key, value)
                }
            end
        '
    }

will get you

           "Subproc" => 0,
"TerminatedNormally" => true,
   "EventTypeNumber" => 5,
            "MyType" => "JobTerminatedEvent",
          "@version" => "1",
       "ReturnValue" => 0,
   "TotalLocalUsage" => "Usr 0 00:00:00, Sys 0 00:00:00",
"TotalReceivedBytes" => 0.0

etc.

[thdesy (Thomas Hartmann)]

ah, many thanks! That is it, what I have been looking for