Elastic Agent Custom Windows Event Logs

Elastic Stack Beats
1

Hello everyone,

I am struggling to add custom Windows Event Logs to my Elastic-Agents / my Agent-Policy and make it work. For example I would like to ingest Elasticsearch with Event Logs from the Serverrole Remotedesktopservices, which uses a few different Log-Channels from Microsoft-Windows-TerminalServices.

I am new to the Elastic Stack and started with winlogbeat as a standalone shipper before trying out fleet and elastic-agents. With Winlogbeat I could add custom logs with either xml-queries or just the channel reference with "- name: ..." itself. This does not work with the custom event logs integration and I cannot comprehend why. I got error Logs that say, that the specified channel was not found, which is not correct.

I tried adding just one channel in the Windows Event Logs integration configuration like "Microsoft-Windows-TerminalServices-SessionBroker-Client/Operational" or editing the .yml file in the advanced options like i did with winlogbeat but I receive no logs.

System Metrics and Standard Windows Channel logs from the System Integration are shipped without any issues.

Working in winlogbeat but not the integration:

winlogbeat.event_logs:

  #Remotedesktop
  - name: Microsoft-Windows-TerminalServices-RemoteConnectionManager/Admin
  #- name: Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational
  - name: Microsoft-Windows-TerminalServices-PnPDevices/Admin
  - name: Microsoft-Windows-TerminalServices-PnPDevices/Operational
  - name: Microsoft-Windows-TerminalServices-SessionBroker-Client/Admin
  - name: Microsoft-Windows-TerminalServices-SessionBroker-Client/Operational
  - name: Microsoft-Windows-RemoteApp and Desktop Connections/Admin
  - name: Microsoft-Windows-TerminalServices-ServerUSBDevices/Operational
  - name: Microsoft-Windows-TerminalServices-ServerUSBDevices/Admin
  - name: Microsoft-Windows-TerminalServices-LocalSessionManager/Operational
  - name: Microsoft-Windows-TerminalServices-LocalSessionManager/Admin
  - name: Microsoft-Windows-TerminalServices-ClientUSBDevices/Operational
  - name: Microsoft-Windows-TerminalServices-ClientUSBDevices/Admin

Also I am not quite sure how I can troubleshoot this more. I know with winlogbeat as a standalone shipper there was the option to start the .exe file with the -e parameter to see if any Handles could not start (In case specified log-Channels were not found or something).

Elastic Stack is in Version 7.17.2 as are the Elastic-Agents.

Thanks for any help in advance.

2

The first thing I would do is to grab a copy of the Agent policy. You'll find the generated config from the winlog integration in there. See Elastic Agent policies | Fleet and Elastic Agent Guide [8.2] | Elastic. The winlog input config should be similar to what Winlogbeat uses. You can share that section here if you'd like. Maybe we can spot a config issue.

The next thing you can do is view the logs from Filebeat (the event log reading code from Winlogbeat is in Filebeat). See View Elastic Agent logs in Fleet | Fleet and Elastic Agent Guide [8.2] | Elastic and in this case you want to view the elastic_agent.filebeat dataset.

3

Thank you for the help, I appreciate it. Currently I have 1 Agent-Policy for Windows Server with 4 integrations - 3 of them are Custom Windows Event Logs:

  - id: winlog-winlogs-47a9ed18-9c18-4f3d-abc2-c8229e2b48a6
    name: winlog-1-Active Directory-Zertifikatdienste
    revision: 4
    type: winlog
    use_output: default
    meta:
      package:
        name: winlog
        version: 1.5.0
    data_stream:
      namespace: windows
    streams:
      - id: winlog-winlog.winlog-47a9ed18-9c18-4f3d-abc2-c8229e2b48a6
        name: Application
        data_stream:
          dataset: winlog.winlog
        xml_query: |
          <QueryList>
            <Query Id="0" Path="Application">
              <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-CertificationAuthority']]]</Select>
              <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-NetworkDeviceEnrollmentService']]]</Select>
              <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-OnlineResponderWebProxy']]]</Select>
              <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-OnlineResponder']]]</Select>
              <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-OnlineResponderRevocationProvider']]]</Select>
              <Select Path="Microsoft-Windows-EnrollmentPolicyWebService/Admin">*[System[Provider[@Name='Microsoft-Windows-EnrollmentPolicyWebService']]]</Select>
              <Select Path="Microsoft-Windows-EnrollmentWebService/Admin">*[System[Provider[@Name='Microsoft-Windows-EnrollmentWebService']]]</Select>
            </Query>
          </QueryList>
        condition: '${host.platform} == ''windows'''
        ignore_older: 72h
        tags:
          - Active Directory-Zertifikatdienste

    name: winlog-3-Netzwerkrichtlinien- und Zugriffsdienste
    revision: 2
    type: winlog
    use_output: default
    meta:
      package:
        name: winlog
        version: 1.5.0
    data_stream:
      namespace: windows
    streams:
      - id: winlog-winlog.winlog-8b3f2f98-8d9d-4bd8-864c-67bd7f2524cf
        name: System
        data_stream:
          dataset: winlog.winlog
        condition: '${host.platform} == ''windows'''
        ignore_older: 72h
        winlogbeat.event_logs:
          - id: Netzwerkrichtlinien- und Zugriffsdienste
            xml_query: |
              <QueryList>
                <Query Id="0" Path="System">
                  <Select Path="System">*[System[Provider[@Name='NPS']]]</Select>
                  <Select Path="Security">*[System[Provider[@Name='Microsoft-Windows-Security-Auditing'] and Task = 12552]]</Select>
                </Query>
              </QueryList>
        tags:
          - Netzwerkrichtlinien- und Zugriffsdienste

  - id: winlog-winlogs-c1e4d4c1-d3f1-49d9-8b6d-1338519d01df
    name: winlog-2-Remotedesktopdienste
    revision: 13
    type: winlog
    use_output: default
    meta:
      package:
        name: winlog
        version: 1.5.0
    data_stream:
      namespace: windows
    streams:
      - id: winlog-winlog.winlog-c1e4d4c1-d3f1-49d9-8b6d-1338519d01df
        name: Remotedesktopdienste
        data_stream:
          dataset: winlog.winlog
        condition: '${host.platform} == ''windows'''
        ignore_older: 72h
        include_xml: true
        winlogbeat.event_logs:
          - name: Microsoft-Windows-TerminalServices-RemoteConnectionManager/Admin
          - name: Microsoft-Windows-TerminalServices-PnPDevices/Admin
          - name: Microsoft-Windows-TerminalServices-PnPDevices/Operational
          - name: Microsoft-Windows-TerminalServices-SessionBroker-Client/Admin
          - name: >-
              Microsoft-Windows-TerminalServices-SessionBroker-Client/Operational
          - name: Microsoft-Windows-RemoteApp and Desktop Connections/Admin
          - name: Microsoft-Windows-TerminalServices-ServerUSBDevices/Operational
          - name: Microsoft-Windows-TerminalServices-ServerUSBDevices/Admin
          - name: Microsoft-Windows-TerminalServices-LocalSessionManager/Operational
          - name: Microsoft-Windows-TerminalServices-LocalSessionManager/Admin
          - name: Microsoft-Windows-TerminalServices-ClientUSBDevices/Operational
          - name: Microsoft-Windows-TerminalServices-ClientUSBDevices/Admin
        tags:
          - Remotedesktopdienste

Also the following errors of one Agent from the elastic_agent.filebeat dataset from a Windows Server with a NPS and RemoteDesktopGateway Role:

[elastic_agent.filebeat][error] 2 errors: Error creating runner from config: Failed to create new event log. failed unpacking config. 1 error: xml_query cannot be used with 'name' accessing config; Error creating runner from config: Failed to create new event log. failed unpacking config. 1 error: xml_query cannot be used with 'name' accessing config
[elastic_agent.filebeat][error] Input 'winlog' failed with: input.go:130: input winlog-winlog.winlog-c1e4d4c1-d3f1-49d9-8b6d-1338519d01df failed (id=winlog-winlog.winlog-c1e4d4c1-d3f1-49d9-8b6d-1338519d01df)
	failed to open windows event log: The specified Channeln was not found. Check the Channel configuration.
[elastic_agent.filebeat][error] failed applying config blocks: <nil>

So I guess xml Queries from the Windows Event log can not be put as 1:1 copy? I am still not sure why the specified channel for the remotedesktop services are not working.

If I can supply any more logs please let me know.

Thank you.

4

Did anyone had a similar issue with the Custom Windows Event Logs Integration or knows what I am doing wrong?

5

So my custom integrations with xml-queries for Certificate Services as well as NPS Logs are working fine and the events are send to my index in Elasticsearch. I am still testing with the custom integration for RDP logs and currently my Agent-Policy for those 3 integrations looks as follows:

- id: winlog-winlogs-47a9ed18-9c18-4f3d-abc2-c8229e2b48a6
name: winlog-1-Active Directory-Zertifikatdienste
revision: 5
type: winlog
use_output: default
meta:
  package:
    name: winlog
    version: 1.5.0
data_stream:
  namespace: windows
streams:
  - id: winlog-winlog.winlog-47a9ed18-9c18-4f3d-abc2-c8229e2b48a6
    name: Application
    data_stream:
      dataset: winlog.winlog
    xml_query: |
      <QueryList>
        <Query Id="0" Path="Application">
          <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-CertificationAuthority']]]</Select>
          <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-NetworkDeviceEnrollmentService']]]</Select>
          <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-OnlineResponderWebProxy']]]</Select>
          <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-OnlineResponder']]]</Select>
          <Select Path="Application">*[System[Provider[@Name='Microsoft-Windows-OnlineResponderRevocationProvider']]]</Select>
          <Select Path="Microsoft-Windows-EnrollmentPolicyWebService/Admin">*[System[Provider[@Name='Microsoft-Windows-EnrollmentPolicyWebService']]]</Select>
          <Select Path="Microsoft-Windows-EnrollmentWebService/Admin">*[System[Provider[@Name='Microsoft-Windows-EnrollmentWebService']]]</Select>
        </Query>
      </QueryList>
    condition: '${host.platform} == ''windows'''
    tags:
      - Active Directory-Zertifikatdienste
- id: winlog-winlogs-c1e4d4c1-d3f1-49d9-8b6d-1338519d01df
name: winlog-2-Remotedesktopdienste
revision: 24
type: winlog
use_output: default
meta:
  package:
    name: winlog
    version: 1.5.0
data_stream:
  namespace: windows
streams:
  - id: winlog-winlog.winlog-c1e4d4c1-d3f1-49d9-8b6d-1338519d01df
    name: Microsoft-Rdms-UI/Admin
    data_stream:
      dataset: winlog.winlog
    xml_query: |
      <QueryList>
        <Query Id="0" Path="Microsoft-Rdms-UI/Admin">
          <Select Path="Microsoft-Rdms-UI/Admin">*</Select>
          <Select Path="Microsoft-Rdms-UI/Operational">*</Select>
          <Select Path="Remote-Desktop-Management-Service-Admin">*</Select>
          <Select Path="Remote-Desktop-Management-Service-Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-SessionBroker-Client/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-SessionBroker-Client/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-RemoteConnectionManager/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-PnPDevices/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-PnPDevices/Operational">*</Select>
          <Select Path="Microsoft-Windows-RemoteApp and Desktop Connections/Admin">*</Select>
          <Select Path="Microsoft-Windows-RemoteApp and Desktop Connection Management/Admin">*</Select>
          <Select Path="Microsoft-Windows-RemoteApp and Desktop Connection Management/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-SessionBroker/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-SessionBroker/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-TSV-VmHostAgent/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-TSV-VmHostAgent/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-ServerUSBDevices/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-ServerUSBDevices/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-ClientUSBDevices/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-ClientUSBDevices/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-RDPClient/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-Licensing/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-Licensing/Operational">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-Gateway/Admin">*</Select>
          <Select Path="Microsoft-Windows-TerminalServices-Gateway/Operational">*</Select>
        </Query>
      </QueryList>
    condition: '${host.platform} == ''windows'''
    event_id: -261
    tags:
      - Remotedesktopdienste
- id: winlog-winlogs-8b3f2f98-8d9d-4bd8-864c-67bd7f2524cf
name: winlog-3-Netzwerkrichtlinien- und Zugriffsdienste
revision: 5
type: winlog
use_output: default
meta:
  package:
    name: winlog
    version: 1.5.0
data_stream:
  namespace: windows
streams:
  - id: winlog-winlog.winlog-8b3f2f98-8d9d-4bd8-864c-67bd7f2524cf
    name: System
    data_stream:
      dataset: winlog.winlog
    xml_query: |
      <QueryList>
        <Query Id="0" Path="System">
          <Select Path="System">*[System[Provider[@Name='NPS']]]</Select>
        </Query>
      </QueryList>
    condition: '${host.platform} == ''windows'''
    tags:
      - Netzwerkrichtlinien- und Zugriffsdienste

I am still getting the same error message and I do not understand why to be honest.

14:40:57.556
elastic_agent.filebeat
[elastic_agent.filebeat][error] Error creating runner from config: Failed to create new event log. failed unpacking config. 1 error: xml_query cannot be used with 'name' accessing config
14:40:57.556
elastic_agent.filebeat
[elastic_agent.filebeat][error] 3 errors: Error creating runner from config: Failed to create new event log. failed unpacking config. 1 error: xml_query cannot be used with 'name' accessing config; Error creating runner from config: Failed to create new event log. failed unpacking config. 1 error: xml_query cannot be used with 'name' accessing config; Error creating runner from config: Failed to create new event log. failed unpacking config. 1 error: xml_query cannot be used with 'name' accessing config
14:40:57.557
elastic_agent.filebeat
[elastic_agent.filebeat][error] failed applying config blocks: <nil>
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.