Decode_xml not working

Elastic Stack Beats
1

Hello,

I'm using winlogbeats version 7.12 and it works great but I have a question.

In my windows log in message field I have an XML. What I would like to do is first check if in that XML in specific field there is a specified text and if yes then only store a subset of those XML fields.

What I do right now is that I'm using processor drop_event with condition: when -> not -> contains -> message: myStringToSearchFor

And then I have a processor script
sth like:

    function process(event) {
                    var re = /<ResponseStatus>\w+<\/ResponseStatus>/;
                    var str = event.Get("message");
                    var myMatch = str.match(re);
                    if(myMatch){
                        event.Put("status", myMatch[0]);
                    }
                    event.Delete("message");
                }

It is only an example with only one field (ResponseStatus) from my xml message extracted, in the future I would like to extract like 5-10 fields from it.

Still this code works quite well, but I'm not fully convinced that it will perform super fast.

I found decode_xml processor but when I put there configuration from documentation it didn't work with informatiion: parameter fields is missing.

        - decode_xml:
    			fields: message
    			target_field: xml
    			ignore_missing: true
    			ignore_failure: true

So question is: do I use decode_xml in an inccorect way? If it would work would it be faster?

2

Using the processor would surely be more performant, especially if you planned on using regex.

Usually I tend to just format the whole message into something that is easier to transform, delete or modify, so if we could get the decode_xml processor to decode the message first it would surely be easier to manage afterwards.

I think what you want to use is field and not fields, for example:

  - decode_xml:
      field: message
      target_field: xml
      ignore_missing: true
      ignore_failure: true

Also make sure the indentation is correct, and that should resolve it.

Hopefully that fixes it :slight_smile:

3

Hello @Marius_Iversen

I tried it end then I get message:

Exiting: Failed to create new event log. missing fields option in winlogbeat.event_logs.0.processors.2.decode_xml
4

Could you post a copy of the whole input + the decode_xml processor? There seems to be an issue then with indentation or similar.

5

Hello @Marius_Iversen

Sure, this is taht file, if I remove decode_xml processor segment then it works so it has to be a probem with decode_xml processor.

winlogbeat.event_logs:
  - name: test
    ignore_older: 1h
    processors:
      - drop_fields:
            fields: ["host", "log", "event", "winlog"]
      - drop_event:
            when:
                not:
                    contains:
                        message: StatusCode=200 OK
      - decode_xml:
            field: message
            target_field: xml
            ignore_missing: true
            ignore_failure: true
      
setup.template.settings:

setup.kibana:

output.file:
  path: "C:/tmp/test_log_new"
  filename: winlogbeat
  rotate_every_kb: 10000
  number_of_files: 1000
6

Hello @witkacy , seems like this was indeed a bug from our side, and this is being resolved in 7.13 and 7.12.2, unfortunately you will have to wait just a little bit longer, sorry for the inconvenience! :frowning: