Winlogbeat - no winlog.logon.id

The winlogbeat 7.7.1 dashboards reference winlog.logon.id.These two threads also mention it and imply it should be in versions > 7.4.
https://discuss.elastic.co/search?q=winlog.logon.id
However, I am running 7.6.2 and 7.7.1 and am not seeing that field in any log, across 100's of machines. My config is pretty basic, basically this:

winlogbeat.event_logs: 
  - name: Application 
    ignore_older: 72h 
  
  - name: System 
  
  - name: Security 
    processors: 
      - script: 
          lang: javascript 
          id: security 
          file: ${path.home}/module/security/config/winlogbeat-security.js 
  
  - name: Microsoft-Windows-Sysmon/Operational 
    processors: 
      - script: 
          lang: javascript 
          id: sysmon 
          file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js 
  
#==================== Elasticsearch template settings ==========================  
  
setup.template.settings: 
  index.number_of_shards: 1 
  
#================================ Outputs =====================================  
  
#----------------------------- Logstash output -------------------------------- 
output.logstash: 
  # The Logstash hosts 
  hosts: ["localhost:5044"] 
  
#================================ Processors ===================================== 
  
# Configure processors to enhance or manipulate events generated by the beat. 
  
processors: 
  - add_host_metadata: ~ 
  - add_cloud_metadata: ~ 
  - add_docker_metadata: ~ 

So where is winlog.logon.id? Is that field currently populated by winlogbeat or not? If so, where is the discrepancy?

I think I now understand how the ${path.home}/module/security/config/winlogbeat-security.js integration works. I think I still have an issue though. Event code 4634, says it is covered by winlogbeat-security.js, but those events are not showing a winlog.logon.id but are showing things like user.name so I believe they are being processed.
The convert looks like {from: "winlog.event_data.TargetLogonId", to: "winlog.logon.id"},
and I have values in kibana for winlog.event._data.TargetLogonId, so it seems like I should have winlog.logon.id as well (or instead), but I dont.

Did you ever find a solution for this issue? Prebuilt visualizations using the Security Module error out for me with the following error: " [Could not locate that index-pattern-field (id: winlog.login.id)]" Its very frustrating to say the least.

I've now stumbled over this discuss entry here.
So I did a short test with versions 7.7.0, 7.7.1 and 7.8.0 started as a service or started from command line with outputs directly to elasticsearch and through logstash.
In every case I've seen values.
Let's first check if entries are existing which should filter out path winlog.logon:

GET winlogbeat-*/_search?filter_path=hits.hits._source.winlog.logon,hits.total.value
{
  "query": {
    "match_phrase": {
      "event.code": "4634"
    }
  }
}

I checked Windows 10. What exact OS are you using?

Do you maybe use WEC (Windows Event Collector)?
So my question is if you can give a bit more metadata around this topic.

And finally I can see the events in the Dashboard as well [Winlogbeat] Overview.

Hello Andre-

My pipeline is the same as yours, I believe: Winlogbeat-->Filebeat-->Logstash

When I run the same query as yours I get back a very different result:

{
   "hits" :  {
      "total" : {
         "value" : 0
        }
    }
}

I am running Server 2016. Output from Winlogbeat Overview Dashboard looks good (see attached). I have not explicitly configured Windows Event Collector and if its functional its using default settings. According to the Services MMC WEC it is active and running.

I have also observed that events that should show up in certain visualizations are not. For instance, if I create and then delete a user, those actions do not show up in the appropriate visualizations.

Ok, the event type id 4634 we were talking about was logging off messages of a particular user


and like you can see this one is documented to be
https://www.elastic.co/guide/en/beats/winlogbeat/master/winlogbeat-module-security.html

Now Deleting a certain user should be events 4720 and 4726 which at least should come up:

But I would like to come back to your architectural answer. I understand we are talking about a server where you host winlogbeat is not the one you are interested in to see the events, as you are talking about WEC.

If I understand you correctly you have at least 2 computers in place:

unknown server or workstation with configured WEF (Windows Event Forwarder)
-->
Windows Server 2016 with configured WEC (Windows Event Collector) and hosting Winlogbeat
events are in services.msc in Folder ForwardedEvents
-->
Logstash
-->
Elasticsearch

This is how I understand your architecture. When you are in ForwardedEvents in services.msc on your WEC Windows2016 Server can you filter for events 4720, 4726 or 4634?

Hello Andre, confirmed, neither creating or deleting user accounts were appearing. It's entirely possible that I have my environment setup incorrectly since I am new to ES. I am going to revert to the simplest config possible and ship everything from Beats directly to ES (and leave out Logstash) and see if I can get it to behave more predictably. Currently Filebeat is processing output for the Filebeat System module, Fortinet and Winlogbeat and all is being written to a single index. Is that even a recommended approach?

I just checked again. We only use winlogbeat, no WEC. I see event.code:4634 with no corresponding winlog.login.id for

  • Windows Server 2008 R2 Standard
  • Windows 10 Pro (10.0.18362.900 (WinBuild.160101.0800)) / 10.0.18362.778 (WinBuild.160101.0800), 10.0.18362.904 (WinBuild.160101.0800),
  • Windows Server 2019 Standard
  • Windows Server 2012 Standard

Are you saying you see winlog.login.id fields?

@minorsatellite
I wouldn't feed winlogbeat data into filebeat indices.
https://www.elastic.co/guide/en/beats/winlogbeat/current/elasticsearch-output.html#index-option-es
default for index are winlogbeat indices winlogbeat-%{[agent.version]}-%{+yyyy.MM.dd} so I assume you went away from default index which means, if you output to filebeat, there are existing ingest pipelines that match this particular index and I have never tested what happens in this case. Would you mind doing what you said going back to the default output elasticsearch settings?

What do you get if you feed into winlogbeat indices and check after some time if you then find data?
GET winlogbeat-*/_search?size=2000&filter_path=hits.hits._source.winlog.logon,hits.total.value

@mgotechlock
Yes I am seeing these fields in logon section:
GET winlogbeat-*/_search?size=2000&filter_path=hits.hits._source.winlog.logon,hits.total.value
Do you see the id field for other resultsets?
Please keep in mind that these fields have the name winlog.logon not login, because in last comment you mentioned login.id, but it's logon.id
https://www.elastic.co/guide/en/beats/winlogbeat/current/exported-fields-security.html
Here the example resultset:

{
  "hits" : {
    "total" : {
      "value" : 10000
    },
    "hits" : [
      {
        "_source" : {
          "winlog" : {
            "logon" : {
              "id" : "0x3e7",
              "type" : "Service"
            }
          }
        }
      },
      {
        "_source" : {
          "winlog" : {
            "logon" : {
              "id" : "0x3e7"
            }
          }
        }
      },
      {
        "_source" : {
          "winlog" : {
            "logon" : {
              "id" : "0x3e7",
              "type" : "Service"
            }
          }
        }
      },
      {
        "_source" : {
          "winlog" : {
            "logon" : {
              "id" : "0x3e7"
            }
          }
        }
      },
      {
        "_source" : {
          "winlog" : {
            "logon" : {
              "id" : "0x3e7",
              "type" : "Service"
            }
          }
        }
      },
      {
        "_source" : {
          "winlog" : {
            "logon" : {
              "id" : "0x3e7"
            }
          }
        }
      },
      {
        "_source" : {
          "winlog" : {
            "logon" : {
              "id" : "0x3e7",
              "type" : "Service"
            }
          }
        }
      },
...

So my issue is not an issue after all. Turns out, unbeknownst to me, someone filtered out some events so they never make it to Elastic and Kibana cant see them.

Ok so issue of @mgotechlock is solved I assume.

@minorsatellite
What about you? Any news here?

I reached out to you on Slack Channel to say that the situation is as before, still receiving the error:
Could not locate that index-pattern-field (id: [winlog.logon.id](http://winlog.logon.id/))

I am actually outputting to a separate Winlogbeat index, if that is what you are asking.

GET winlogbeat-*/_search?size=2000&filter_path=hits.hits._source.winlog.logon,hits.total.value

{
  "hits" : {
    "total" : {
      "value" : 10000
    }
  }
}

Here is the Logstash pipeline I am using:

input {
        beats {
                port => 5044
        }
}
filter {
        if [type] == "syslog" {
                grok {
                        match => { "message" => "%{SYSLOGLINE}" }
                }
                date {
                        match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
                }
        }
}
output {

        if [event][module] == "fortinet" {
                elasticsearch {
                #ssl => true
                #ssl_certificate_verification => true
                #cacert => ["elastic\ca.crt"]
                hosts => ["10.101.10.25:9200"]
                #user => "${ES_USR}"
                #password => "${ES_PWD}"
                manage_template => false
                ilm_rollover_alias => "filebeat-7.8.0-fortinet"
                ilm_policy => "filebeat-fortinet"
                ilm_pattern => "{now/d}-000001"
                pipeline => "%{[@metadata][pipeline]}"
        }

        #    stdout {
        #               codec => json
        #       }
        }
        else {
                elasticsearch {
                hosts => ["10.101.10.25:9200"]
                #user => some_user
                #password => some_password
                manage_template => false
                index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
                }
            }

}

In the winlogbeat-security.js file, there are numerous field renaming (conversion) actions taking place, in the form of:

{from: "winlog.event_data.SubjectLogonId", to: "winlog.logon.id"},

Are these correct, and if so, what should be happening? Are any modifications required here to remedy the errors?

unknown server or workstation with configured WEF (Windows Event Forwarder)
-->
Windows Server 2016 with configured WEC (Windows Event Collector) and hosting Winlogbeat
events are in services.msc in Folder ForwardedEvents
-->
Logstash
-->
Elasticsearch

Not exactly, I have 2 Winlogbeat servers are shipping their logs to a single node ES cluster, outputting first to Filebeat, then Logstash, then to ES. Again, I aim not too sure about WEC configuration, and WEF is configured in my YAML file.

I think the main question is if your winlogbeat.yml is configured to care about the ForwardedEvents Folder In Event Log and the underlying js fields:

It should be something like this

....

 - name: Security
    processors:
      - script:
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js

  - name: Microsoft-Windows-Sysmon/Operational
    processors:
      - script:
          lang: javascript
          id: sysmon
          file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js


###beginning###
  - name: ForwardedEvents
    tags: [forwarded]
    processors:
      - script:
          when.equals.winlog.channel: Security
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js
      - script:
          when.equals.winlog.channel: Microsoft-Windows-Sysmon/Operational
          lang: javascript
          id: sysmon
          file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js
 ###end###

Let me know how it goes.

1 Like

Actually, the main question is why is winlog.login.id field missing from my index pattern :slight_smile:

Comparing my YAML file to your code snippet, they look exactly the same.

1 Like

Guess this is turning out to be a bug with the WLB code.

Ok, great find. But the question that is still open for me is, do you have only an issue with the field missing in the dashboard or do you have the situation as well that you don’t have any content in winlog.logon.id field?

Ok,
@minorsatellite
if I understand correctly and looking to slack it seems that everything is working for you now?