Double escaping in expected due to windows events

nahuel978 · April 4, 2024, 5:03am

Hello everyone! I was studying the system/security decoder and I noticed that in the expected ones, the paths of records or files are with double backslashes. I understand that it is to escape the bar that already brings the windows event. But here comes my doubt. What event do they take? the overview event or the xml event? because the xml event already comes with the escape, so the output in elastic should have 4 backslashes instead of 2. unless it is not the xml event that is being taken. I have this doubt since I saw that the events are in evtx format. example: integrations/packages/system/data_stream/security/_dev/test/pipeline/test-4670-windowssrv2016.json at 3d69d50f4c2aede1e9b7472485a6a06d8619d702 · elastic/integrations · GitHub

lesio · April 4, 2024, 10:34am

Winlogbeat is unmarshalling the xml

Are you sure you're reading/viewing the xml data correctly?

nahuel978 · April 4, 2024, 12:26pm

Hi lesio! The confusion arises when you see how the events are in your xml view:

I noticed that in its general view the paths come out with a backslash and in the xml view they come out with two backslashes (due to the escape character). When transforming this into json, shouldn't it be escaped again to preserve the two backslashes, leaving \\ instead of \?

Yamin_Tian · April 4, 2024, 5:14pm

Anyone can correct me. I did some digging on the source code, it seems in transforming the event into json, winlogbeat isn't trying to preserve the format of raw XML input, rather, it preserves the information.
This should be the UnmarshalXML function Leszek mentioned:

github.com

elastic/beats/blob/master/winlogbeat/eventlog/wineventlog.go#L346


      
          			return nil, 0, errors.Wrap(err, "failed to recover from RPC_S_INVALID_BOUND")
          		}
          		return l.eventHandles(maxRead / 2)
          	default:
          		incrementMetric(readErrors, err)
          		logp.Warn("%s EventHandles returned error %v", l.logPrefix, err)
          		return nil, 0, err
          	}
          }
          
          func (l *winEventLog) buildRecordFromXML(x []byte, recoveredErr error) (Record, error) {
          	includeXML := l.config.IncludeXML
          	e, err := winevent.UnmarshalXML(x)
          	if err != nil {
          		e.RenderErr = append(e.RenderErr, err.Error())
          		// Add raw XML to event.original when decoding fails
          		includeXML = true
          	}
          
          	err = winevent.PopulateAccount(&e.User)
          	if err != nil {

which eventually calls into this decode function to decode the raw XML:

github.com

elastic/beats/blob/main/libbeat/common/encoding/xml/decode.go#L55


      
          
          // LowercaseKeys causes the Decoder to transform all key name to lowercase.
          func (d *Decoder) LowercaseKeys() { d.lowercaseKeys = true }
          
          // Decode reads XML from the input stream and return a map containing the data.
          func (d *Decoder) Decode() (map[string]interface{}, error) {
          	_, m, err := d.decode(nil)
          	return m, err
          }
          
          func (d *Decoder) decode(attrs []xml.Attr) (string, map[string]interface{}, error) {
          	elements := map[string]interface{}{}
          	var cdata string
          
          	for {
          		t, err := d.xmlDec.Token()
          		if err != nil {
          			if err == io.EOF { //nolint:errorlint // io.EOF should never be wrapped and is not by xml.Decoder.
          				return "", elements, nil
          			}
          			return "", nil, err

It seems when dealing with Windows paths, the goal is to be able to locate the original paths from the json. I believe the path strings we see in json is expected.

lesio · April 5, 2024, 1:56pm

I see, honestly I didn't even know the xml view exist in Kibana.

It is somewhat confusing that in some places the paths must be escaped and in others not. I stepped on this problem a few times, but I forgot where it was.