# Winlogbeats xml\_query support for new line chars

**URL:** <https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005>\
**Category:** Beats\
**Tags:** winlogbeat\
**Created:** [November 27, 2023, 3:43am UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005 "2023-11-27T03:43:41Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![grants](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grants/32/117040_2.png) [@grants](https://discuss.elastic.co/u/grants)\
**Post date:** [November 27, 2023, 3:43am UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005/1 "2023-11-27T03:43:41Z")

</div>

Running winlogbeat version 8.11.1 and trying to use a custom xml\_query for the event logs. My issue is the data I need to match contains a new line sequence that is proving difficult to filter for.

Provided here in an include example of what I am trying to match, once I get the match working, I can apple this to an exclude filter I'm actually trying to make to ignore all the powershell noise.

```auto
---
winlogbeat.event_logs:
  - id: new-line-test
    fields:
      version: 0
    xml_query: |
      <QueryList>
        <Query Id="0" Path="Microsoft-Windows-PowerShell/Operational">
          <Select Path="Microsoft-Windows-PowerShell/Operational">
            *[System[EventID=4103]] and
            *[EventData[Data[@Name='Payload']='CommandInvocation(Out-Default): "Out-Default"&#13;&#10;']]
          </Select>
        </Query>
      </QueryList>
output.kafka:
  hosts: ["kafka:9092"]
  topic: winlogbeat

```

Notice the hex encoded chars for `\r\n` this filter works just fine in Event Viewer

These escaped new line chars do not work in PowerShell and if you use wevtutil this would be the corresponding structured query that would work:

```xml
<QueryList>
  <Query Id="0" Path="Microsoft-Windows-PowerShell/Operational">
    <Select Path="Microsoft-Windows-PowerShell/Operational">
      *[System[EventID=4103]] and
      *[EventData[Data[@Name='Payload']='CommandInvocation(Out-Default): "Out-Default"
']]
    </Select>
  </Query>
</QueryList>

```

I've tried escaped `\r\n`, I've tried literal new-line in my yaml file, and I've even tried `\n` hoping it would just work.

Any ideas how to get a structured query like this to work? Would this be considered a bug? There has to be something with the `>` or `|` scalar formatting and the xml nested that is causing a problem, but I can't get any of it to work.

I'd like to figure this out so I can use a single query for some of my logs and have the query itself filter out the noise instead of reading it all and dropping it in a processor. Main reason for this is the log volume I'm reading is pretty high and results in higher than expected resource utilization from Winlogbeat. I tend to read then drop nearly 90% of the logs.

---

<div class="post-metadata">

**Author:** ![grants](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grants/32/117040_2.png) [@grants](https://discuss.elastic.co/u/grants)\
**Post date:** [November 27, 2023, 3:49am UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005/2 "2023-11-27T03:49:26Z")

</div>

If needed, this is how you would test the powershell version:

```ps1
wevtutil qe /sq:true /c:1 <path to xml file>

```

I'd expect any structured query that works for PowerShell should also work in the winlogbeat.yml config as a `xml_query` setting, but this doesn't seem to be the case with these issues over new lines.

---

<div class="post-metadata">

**Author:** ![grants](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grants/32/117040_2.png) [@grants](https://discuss.elastic.co/u/grants)\
**Post date:** [November 27, 2023, 3:53am UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005/3 "2023-11-27T03:53:19Z")

</div>

This specific xml\_query setting will pass syntax checks and matches the powershell version, but will not actually read any events.

```auto
    xml_query: |
        <QueryList>
          <Query Id="0" Path="Microsoft-Windows-PowerShell/Operational">
            <Select Path="Microsoft-Windows-PowerShell/Operational">
              *[System[EventID=4103]] and
              *[EventData[Data[@Name='Payload']='CommandInvocation(Out-Default): "Out-Default"
        ']]
            </Select>
          </Query>
        </QueryList>

```

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [November 27, 2023, 4:29pm UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005/4 "2023-11-27T16:29:00Z")

</div>

Assuming the `\r` is required for the match then I think you were are on the right path by doing the escaping. But with YAML the escaped characters can only be used with the double-quoted string format.

From [YAML&nbsp;Ain’t&nbsp;Markup&nbsp;Language&nbsp;(YAML™) Version&nbsp;1.2](https://yaml.org/spec/1.2-old/spec.html#id2787109)

> This is the only style capable of expressing arbitrary strings, by using “\” escape sequences. This comes at the cost of having to escape the “\” and “"” characters.

```auto
xml_query: "<QueryList>\n <Query Id=\"0\" Path=\"Microsoft-Windows-PowerShell/Operational\">\n <Select Path=\"Microsoft-Windows-PowerShell/Operational\">\n *[System[EventID=4103]] and\n *[EventData[Data[@Name='Payload']='CommandInvocation(Out-Default): \"Out-Default\"\n']]\r\n </Select>\n </Query>\n</QueryList>"

```

If you run these test cases ([Go Playground - The Go Programming Language](https://go.dev/play/p/2UmYiP9MrKx)), you can see in the hex dump output that only the double-quoted version actually contains the 0x0D character.

---

<div class="post-metadata">

**Author:** ![grants](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grants/32/117040_2.png) [@grants](https://discuss.elastic.co/u/grants)\
**Post date:** [November 27, 2023, 11:52pm UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005/5 "2023-11-27T23:52:34Z")

</div>

If I were to slam this all onto 1 line, the spacing between xml elements isn't needed and the double quotes inside most of this can be converted to single quotes.

This works in powershell:

```auto
<QueryList><Query Id='0' Path='Microsoft-Windows-PowerShell/Operational'><Select Path='Microsoft-Windows-PowerShell/Operational'>*[System[EventID=4103]] and *[EventData[Data[@Name='Payload']='CommandInvocation(Out-Default): "Out-Default"
']]</Select></Query></QueryList>

```

If I escape the `"` with `\"` into the yaml config I end up with this:

```auto
<QueryList><Query Id='0' Path='Microsoft-Windows-PowerShell/Operational'><Select Path='Microsoft-Windows-PowerShell/Operational'>*[System[EventID=4103]] and *[EventData[Data[@Name='Payload']='CommandInvocation(Out-Default): \"Out-Default\"
']]</Select></Query></QueryList>

```

And lastly replacing that newline with `\n`

```auto
<QueryList><Query Id='0' Path='Microsoft-Windows-PowerShell/Operational'><Select Path='Microsoft-Windows-PowerShell/Operational'>*[System[EventID=4103]] and *[EventData[Data[@Name='Payload']='CommandInvocation(Out-Default): \"Out-Default\"\n']]</Select></Query></QueryList>

```

This passes a test config, but when running it doesn't actually collect any events like it should.

Example winlogbeat.yml:

```auto
winlogbeat.event_logs:
  - id: new-line-test
    fields:
      version: 0
    #xml_query: "<QueryList><Query Id='0' Path='Microsoft-Windows-PowerShell/Operational'><Select Path='Microsoft-Windows-PowerShell/Operational'>*[System[EventID=4103]]</Select></Query></QueryList>"
    xml_query: "<QueryList><Query Id='0' Path='Microsoft-Windows-PowerShell/Operational'><Select Path='Microsoft-Windows-PowerShell/Operational'>*[System[EventID=4103]] and *[EventData[Data[@Name='Payload']='CommandInvocation(Out-Default): \"Out-Default\"\n']]</Select></Query></QueryList>"

```

The escaped xml\_query was me testing to make sure the quotes were at least right around the xml and that the single quotes worked. The commented out version did collect while the other did not collect anything (it should have).

Maybe the better way is to support a xml\_query reference to a file if new lines are going to be an issue like this. According to the xml docs, it seems I should be able to use `&#10;` but when attempting to use this, I get a query invalid error from winlogbeat, I get the same error when using powershell, but this works with Event Viewer.

I've tried the sample you provided, but also the same result, passes syntax checks, but doesn't collect any events. Would this new line char need some additional escaping to pass through yaml and render with xml?

---

<div class="post-metadata">

**Author:** ![grants](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grants/32/117040_2.png) [@grants](https://discuss.elastic.co/u/grants)\
**Post date:** [November 28, 2023, 12:07am UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005/6 "2023-11-28T00:07:17Z")

</div>

One additional note, I've been using wevtutil to validate these queries with the structuredquery option which is what I thought this setting would support. If I use their standard filtering query option, the new line char never seems to match no matter what I do.

I'm not too sure how Winlogbeat queries for events and if its the same as wevtutil's structuredquery. I also haven't tried the experimental API flag.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [December 4, 2023, 10:41pm UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005/7 "2023-12-04T22:41:53Z")

</div>

Sorry I haven't had time to investigate this further. Could you open a [elastic/beats](https://github.com/elastic/beats/issues/new/choose) issue for this so that we can help find a way to pass those special characters through into Windows.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 2, 2024, 12:41am UTC](https://discuss.elastic.co/t/winlogbeats-xml-query-support-for-new-line-chars/348005/8 "2024-01-02T00:41:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.