Active Directory logs and mapping to ECS (I am stumped)

[iukea (jordan)]

Hello guys/girls,

Need some words of wisdom on ECS and on Active Directory logs.

I am currently logging a Windows server 2012 Domain controller and used to flowing guide (provided by ElasticSearch)
- https://www.elastic.co/blog/monitoring-windows-logons-with-winlogbeat

• https://www.elastic.co/guide/en/beats/winlogbeat/master/exported-fields-ecs.html

Attached is a picture of what the SIEM app sees. (and my goal is to get it so I can see what users are logging in and out and also to follow ElasticSearch's best practices)

My logstash.conf file

input {
  beats {
    port => 5044
  }
}

filter {
  geoip {
    source => "[event_data][IpAddress]"
  }
}

output {
  elasticsearch {
    hosts => "logstash.elastic.local:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

winlogbeat.yml file

winlogbeat.event_logs:
  - name: Security
    ignore_older: 168h

output.logstash:
  hosts: ["logstash.elastic.local:9200"]

[iukea (jordan)]

Ah got some progress

This helped
https://www.elastic.co/guide/en/beats/winlogbeat/7.4/winlogbeat-module-security.html

[iukea (jordan)]

still not fully working...

[andrewkroh]

@iukea

still not fully working...

What's not working? I see you enabled the Security module in Winlogbeat.

There's one enhancement in progress to add some additional event IDs into the Security module. These are mostly related to account management events. https://github.com/elastic/beats/pull/13530

[iukea (jordan)]

Thank you for replying

I am having an issue getting all the Winbeat fields mapped to ECS for the ElasticSearch SIEM app.

(mainly the IP fields, and DNS events) Logging an Active Directory server

I tried setting up a field template called "ecs.yml" which I got from

elastic/ecs/blob/master/generated/beats/fields.ecs.yml

I am also having issues getting GeoIP data to work and tried setting up a pipeline with little luck.

Below is my Winbeat.yml file

winlogbeat.event_logs:
  - name: Application
    processors:
    - add_host_metadata:
        netinfo.enabled: true
        cache.ttl: 5m
  - name: Security
    processors:
      - script:
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js
  - name: System

  - name: Microsoft-Windows-Sysmon/Operational
    processors:
      - script:
          lang: javascript
          id: sysmon
          file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js


winlogbeat.event_logs:
  - name: Application
    processors:
    - add_host_metadata:
        netinfo.enabled: true
        cache.ttl: 5m
  
  - name: Security
    processors:
      - script:
            lang: javascript
            id: security
            file: ${path.home}/module/security/config/winlogbeat-security.js
  - name: System


  - name: Microsoft-Windows-Sysmon/Operational
    processors:
      - script:
          lang: javascript
          id: sysmon
          file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js



processors:
- dns:
   type: reverse
   action: append
   fields:
     server.ip: server.hostname
     client.ip: client.hostname
   success_cache:
     capacity.initial: 1000
     capacity.max: 10000
   failure_cache:
    capacity.initial: 1000
    capacity.max: 10000
    ttl: 1m
   nameservers: ['8.8.8.8']
   timeout: 500ms

setup.template.fields: ${path.home}/ecs.yml
output.logstash: 
  hosts: ["192.168.86.3:6050"]
  pipeline: geo-info

[webmat (Mathieu Martin)]

You don't need to specifically include the ECS field definitions for any Beat, starting with 7.0, unless you want to make an older Beat (e.g. 7.0 which was ECS 1.0.0) forward-compatible with a later version of ECS (e.g. ECS 1.2.0). But in general, each Beat already defines all ECS fields.

Second note on leveraging the ECS generated files. If you have other more custom pipelines (not this Winlogbeat situation), you should not use files from the master branch of the ECS repo. You should make sure to pick the correct Git tag that matches the ECS release you're targeting.

The destination of your reverse DNS lookup should go to client.domain and server.domain, there's no .hostname field in client/server nor source/destination.

The IP addresses are indeed inside event_data, however your Logstash config should look for them at "[winlog][event_data][IpAddress]" (you're missing the winlog nesting). Adjusting this should get you most of the IP addresses.

Note also that depending on the providers / event IDs you consume, I've seen typos in some IpAddress fields. Sometimes it's Ip Address, or other small typos. So adjusting to "[winlog][event_data][IpAddress]" will get you most of the way there. If you still see it missing from time to time, look for a typo, and add that as a fallback to your pipeline, to correctly capture those as well.

[iukea (jordan)]

Thank you!

Quick question about Login and Logout processes (event 4672)
(External reference link https://serverfault.com/questions/800403/a-ton-of-logon-off-events-in-event-viewer)

generates a tone of login noisily activity

is there any way to filter events " For 4624 and 4634 events with logon type 3:" ?