Fields don't show up when events are published

Elastic Stack Beats
1

Hi all,

I'm struggling with a windows event to publish.
Event xml below, strongs fields aren't publish by Winlogbeat.

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name="Microsoft-Windows-CertificateServicesClient-Lifecycle-System" Guid="{bc0669e1-a10d-4a78-834e-1ca3c806c93b}" /> 
  <EventID>1003</EventID> 
  <Version>0</Version> 
  <Level>3</Level> 
  <Task>0</Task> 
  <Opcode>0</Opcode> 
  <Keywords>0x8000000000000000</Keywords> 
  <TimeCreated SystemTime="2021-10-16T16:02:39.728582800Z" /> 
  <EventRecordID>167</EventRecordID> 
  <Correlation /> 
  <Execution ProcessID="140" ThreadID="2436" /> 
  <Channel>Microsoft-Windows-CertificateServicesClient-Lifecycle-System/Operational</Channel> 
  <Computer>server.exemple.local</Computer> 
  <Security UserID="S-0" /> 
  </System>
***- <UserData>***
***- <CertNotificationData ProcessName="taskhostw.exe" AccountName="exemple\server$" Context="Machine">***
***- <CertificateDetails Thumbprint="11d11">***
***  <Template Name="Template Name" OID="1.0.0" /> ***
***- <SubjectNames>***
***  <SubjectName>CN=server.exemple.local</SubjectName> ***
***  <SubjectName>server.exemple.local</SubjectName> ***
***  <SubjectName>server</SubjectName> ***
***  <SubjectName>exemple.local</SubjectName> ***
***  </SubjectNames>***
***- <EKUs>***
***  <EKU Name="Server Authentication" OID="1.3.6.1.5.5.7.3.1" /> ***
***  <EKU Name="Client Authentication" OID="1.3.6.1.5.5.7.3.2" /> ***
***  </EKUs>***
***  <NotValidAfter>2021-10-16T16:07:44Z</NotValidAfter> ***
***  </CertificateDetails>***
***  </CertNotificationData>***
***  </UserData>***
  </Event>

Winlogbeat DEBUG:

2022-01-18T17:45:31.940+0100    DEBUG   [processors]    processing/processors.go:203    Publish event: {
  "@timestamp": "2021-09-24T08:02:33.486Z",
  "@metadata": {
    "beat": "winlogbeat",
    "type": "_doc",
    "version": "7.13.4"
  },
  "event": {
    "code": "1003",
    "kind": "event",
    "provider": "Microsoft-Windows-CertificateServicesClient-Lifecycle-System",
    "created": "2022-01-18T16:45:31.940Z"
  },
  "message": "A certificate is about to expire. Please refer to the \"Details\" section for more information
  "fields": {
    "env": "DEV",
  },
  "log": {
    "level": "warning"
  },
  "host": {
    "name": "server.exemple.com"
  },
  "winlog": {
    "channel": "Microsoft-Windows-CertificateServicesClient-Lifecycle-System/Operational",
    "user": {
      "domain": "NT AUTHORITY",
      "name": "SYSTEM",
      "type": "User",
      "identifier": "S-1-5-18"
    },
    "provider_name": "Microsoft-Windows-CertificateServicesClient-Lifecycle-System",
    "record_id": 75,
    "opcode": "Info",
    "api": "wineventlog",
    "computer_name": "server.exemple.com",
    "provider_guid": "{b}",
    "process": {
      "pid": 4168,
      "thread": {
        "id": 4700
      }
    },
    "event_id": "1003"
  },
  "tags": [
    "type1"
  ],
  "ecs": {
    "version": "1.9.0"
  },
  "agent": {
    "ephemeral_id": "463f0e5c-3deb-4255-8387-7db955fe96d6",
    "id": "30be004e-1fa6-48ab-b276-de6903d55376",
    "name": "SERVER",
    "type": "winlogbeat",
    "version": "7.13.4",
    "hostname": "SERVER"
  }
}

Winlogbeat conf:

winlogbeat.event_logs:
  - name: Microsoft-Windows-CertificateServicesClient-Lifecycle-System/Operational
    event_id: 1002,1003
2

The logs show it's being published. Are u saying that it's not in Elasticsearch? What's ur output config?

3

Hello,

I will give more infos on this topic.
Some events to track certificates expiration/renewal are published by winlogbeat but incomplete.

On the windows event viewer you can see in the XML all the fields.
The event is published, but some fields are missing - they are noted in bold in the UserData section.

The event is published but the interesting fields are not published by the winlogbeat Agent.

Maybe there is something with the declaration of these fields in the fields.yml ; I don't see anything related to the missing fields...

ELK : 7.16.2 + Kafka.
Winlogbeat : 7.13.4

1 Like
5

Thanks for clarification... It is published but incomplete.

6

Hello,

Anyone has an idea to have the missing Fields published by winlogbeat to Elasticsearch ?

The doc Winlogbeat fields | Winlogbeat Reference [7.16] | Elastic is mentioning :
event.original
The raw XML representation of the event obtained from Windows. This field is only available on operating systems supporting the Windows Event Log API (Microsoft Windows Vista and newer). This field is not included by default and must be enabled by setting include_xml: true as a configuration option for an individual event log. The XML representation of the event is useful for troubleshooting purposes. The data in the fields reported by Winlogbeat can be compared to the data in the XML to diagnose problems

The missing fields are not shipped in Elastic , but are present in the XML :

- <UserData>
- <CertNotificationData ProcessName="taskhostw.exe" AccountName="exemple\server$" Context="Machine">
- <CertificateDetails Thumbprint="11d11">
  <Template Name="Template Name" OID="1.0.0" /> 
- <SubjectNames>
  <SubjectName>CN=server.exemple.local</SubjectName> 
  <SubjectName>server.exemple.local</SubjectName> 
  <SubjectName>server</SubjectName> 
  <SubjectName>exemple.local</SubjectName> 
  </SubjectNames>
- <EKUs>
  <EKU Name="Server Authentication" OID="1.3.6.1.5.5.7.3.1" /> 
  <EKU Name="Client Authentication" OID="1.3.6.1.5.5.7.3.2" /> 
  </EKUs>
  <NotValidAfter>2021-10-16T16:07:44Z</NotValidAfter> 
  </CertificateDetails>
  </CertNotificationData>
  </UserData>

Any Clue to have the missing fields published ?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.