Missing events from Windows Event Collector (WEC)

Elastic Stack Beats
1

Issue: I am missing at least some events that exist on the WEC in Elasticsearch.

The Environment: I am testing Elastic Stack. At the moment, I have a single Ubuntu server running Logstash, Elasticsearch, and Kibana with 2 vCPU and 8 GB RAM. I have a WEC running Winlogbeat 7.5.2 feeding logs from the ForwardedEvents, Application, Security, and System logs on the WEC into Logstash. On Logstash, I'm using the pipeline configs from the HELK project, with the Kafka input (0002-kafka-input.conf) disabled and the outputs enabled with the correct server destination (localhost) and no authentication.

The Scenario: For example, I triggered a DNS lookup of "ferrari.com" on my workstation. This resulted in 3 event 3008 entries from Microsoft-Windows-DNS-Client/Operational on the WEC. Each of these seems to have a unique "EventRecordID" in the Windows event XML data (8000102, 8000111, 8000123).

I searched in Kibana for these events and it returned 4 results. Oddly, it really seems like two pairs of cloned results (two of them seem identical to each other and have one _id, the other two seem identical to each other and have another _id). In any case, one of the events from the WEC does not appear at all (the one with EventRecordId 8000102).

I'm new to Elastic Stack. I checked the winlogbeat log file and searched for "error" and received no results. What sort of other troubleshooting can I do to determine what happened with the missing event?

Supporting Information:

winlogbeat.yml:

winlogbeat.event_logs:
  - name: ForwardedEvents
    forwarded: true
    tags: ["forwarded"]
    processors:
      - script:
          when.equals.winlog.channel: Security
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js
      - script:
          when.equals.winlog.channel: Microsoft-Windows-Sysmon/Operational
          lang: javascript
          id: sysmon
          file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js

  - name: Application
    ignore_older: 72h

  - name: System
    ignore_older: 72h

  - name: Security
    processors:
      - script:
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:

  host: "kibanahost:5601"

output.logstash:

  hosts: ["logstashhost:5044"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

logging.level: info
logging.to_files: true
logging.files:
  path: C:\Program Files\Winlogbeat\logs
  name: winlogbeat
  rotateeverybytes: 104857600
  keepfiles: 7
logging.json: false
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.