Winlogbeat dashboard incomplete/missing fields

Elastic Stack Beats
1

Hello,

Winlogbeat and ELK 7.10.0

Trying to show a college of mine that visualisation is much easier with Kibana then Splunk, I'm trying to build a use case with Active Directory security logging. But is looks like somewhere in the process I'm doing thing wrong or it is simply not working as expected.
I'm trying to import a evtx file into an ELK stack running on a Linux server. This exported file has been copied to a Windows10 desktop in order to have it ingested via Winlogbeat.

  • Import index into Elasticsearch template via:
    win10> .\winlogbeat.exe export template --es.version 7.10.0 | Out-File -Encoding UTF8 winlogbeat.template.json
    lx> curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/winlogbeat-7.10.0 -d@winlogbeat.template.json

  • Import the dashboards:
    win10> .\winlogbeat.exe setup --dashboards

  • import the evtx file via Logstash beats:
    win10> .\winlogbeat.exe -e -c .\winlogbeat-evtx.yml -E EVTX_FILE=Security.evtx

There is data to explore, but it looks like there are fields missing even though I'me sure the eventid's are present an should be processed by winlogbeat-security.js

Screenshot 2020-11-23 at 16.11.12
Screenshot 2020-11-23 at 16.11.123016×1520 297 KB

Any idea's?

Kind regards,
Andre

2

Hi,
Could you check your mappings and share it here?
Usually when you can't use aggregation on a field is because the field type isn't keyword.
So check your data mapping and make sure it's using the correct template and mapping.

3

Hi,
Attached is the mapping from the index winlogbeat.
This is a fresh empty ELK single node Linux system, so how could it end up using a wrong template?
Thanks for your time!
Andre
PS: not able to attach files? Post is limited by number of characters

https://pastebin.pl/view/30d359c1

4

Take a look at lines 88-95. Your event.action main type is text with a keyword subtype.
As you know, You can't use aggregation on text fields.
Could you share your index template too? because I'm guessing your template mappings is correct and for some reason, your index isn't using your template.

5

Sorry, I don't know. :wink: It is a fresh installation following all the instructions found on the elastic website and that is causing problems because I missed a step or did something wrong? I did not create a dashboard or mapping or template by myself.

6

Oke, when having a look via Index Management I have an indice winlogbeat-2020-11-18, Looking at Index Templates I see a Legacy index template winlogbeat-7.10.0 which will be applied on index pattern winlogbeat-7.10.0-* . So guess you are absolutely right about not using the template. Question is where to correct this.
Should I correct this in the Logstash config?:
output{
elasticsearch {
hosts => ["http://127.0.0.1:9200"]
index => "%{[@metadata][beat]}-%{+YYYY-MM-dd}"
}

7

Glad you could find the problem!

You have two option:

  1. Change the index pattern in your template to match the name of your indices.

  2. Change your index name in Logstash/beats configurations to match your index template pattern.

I recommend the second option.

1 Like
8

Changed Logstash.conf:
input{
beats{
port => "5044"
}
}
output{
elasticsearch {
hosts => ["http://127.0.0.1:9200"]
index => "%{[@metadata][beat]}-7.10.0-%{+YYYY-MM-dd}"
# index => "%{[@metadata][beat]}-%{+YYYY-MM-dd}"
}
}

Removed al previous data and started all over again.
Dashboards are now definitively looking beter, thanks, but still missing something :
Could not locate that index-pattern-field (id: winlog.logon.id)
Trying to find out what's/why still missing, will let you know or maybe you already have a hint where to look for.

9

Sorry, I'm not experienced with logstash. hopefully, someone from elastic would help you.

10

Thanks for your time and pointing out in the right direction!

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.