Troubshooting mapping conflicts after 7.2 upgrade, event id's

[mathurin68]

Upgraded to elasticstack/Winlogbeats 7.2 and it's great! But I seem to have some conflicts in mapping somewhere and I'm not sure how to troubleshoot. In the past I could simply got to 'Index Patterns' hit the 'refresh field list' and everything worked. Now I'm not sure...

It seems to be the fields related to event ID

and

and they seem to be coming in like this in syslog(debugging)...
i.e. event 5860

Jul  8 16:14:09 elkstack logstash[88195]:         "provider_name" => "Microsoft-Windows-WMI-Activity",
Jul  8 16:14:09 elkstack logstash[88195]:              "event_id" => 5860
Jul  8 16:14:09 elkstack logstash[88195]:     },
Jul  8 16:14:09 elkstack logstash[88195]:            "ecs" => {
Jul  8 16:14:09 elkstack logstash[88195]:         "version" => "1.0.0"
Jul  8 16:14:09 elkstack logstash[88195]:     },
Jul  8 16:14:09 elkstack logstash[88195]:          "event" => {
Jul  8 16:14:09 elkstack logstash[88195]:            "kind" => "event",
Jul  8 16:14:09 elkstack logstash[88195]:         "created" => "2019-07-08T16:14:08.363Z",
Jul  8 16:14:09 elkstack logstash[88195]:            "code" => 5860

I re-indexed the logs and they seemed to be OK...
How do I troubleshoot this? Appreciate any help...

[andrewkroh]

What version of Winlogbeat where you running previously?

[mathurin68]

Hey @andrewkroh! Man that's a good question at this point... I want to say 6.8?

[mathurin68]

These 4 -

[mathurin68]

Weird... so it wasn't production data so I cleared ALL the indices.

Set the WEF to use elasticsearch, and used powershell to manually load the template

PS > .\winlogbeat.exe setup --index-template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["ipaddress:9200"]'

Once the template and everything was in place, I re-added logstash to the pipeline but in a very clear fashion..
Went ahead and created the index pattern and was good to go no conflicts...

So I went ahead and added logstash back to the pipeline but with a very simple...

input {

beats {
    port => 5044
  }
}

and an output

output {
  elasticsearch {
    hosts => ["ip_address:9200"]
    user => "elastic"
    password => "password"
    index => "%{[@metadata][beat]}-%{+xxxx.ww}"
  }
}

Now when I refresh the index pattern I created I get 3 conflicts

event.code
winlog.event.id
winlog.record_id

Soooo I had to delete the index pattern I created with just elasticsearch
and recreate one with the pipeline going through logstash.
winlog.event.id was a keyword in one and a long in the other??? I have no idea...

BUT it is all working now. Like my python coding... I have no idea why.

[andrewkroh]

I recommend using the config from our docs for your Logstash output. https://www.elastic.co/guide/en/elastic-stack-get-started/7.2/get-started-elastic-stack.html#logstash-setup

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

Assuming that you always install the index template prior to ingesting data from each new Winlogbeat version then this will ensure that you never run into mapping exceptions at ingest time due to schema changes.

It won't necessarily guarantee there are no conflicts in the Kibana index pattern, but I think there wouldn't be as many as you show above (particular for the new winlog fields which have not changed). You could delete the index pattern then recreate it with winlogbeat.exe setup --dashboards. The old fields that no longer exist won't be listed though. Another option is to delete the old indexes (or close them) and then refresh the index pattern.