Windows integration for event logs error

Elastic Stack Beats
,
1

Hi

I have had a look at the Windows integration to collect events from the Windows event log but I believe there may be a mapping error.

We have an environment where all Windows event logs are forwarded to a central log collector. This device has been running winlogbeat to collect logs from the 'Forwarded Events' channel to Elastic.

Today I tried running this via the Windows integration which has options to collect various log events, however there is an error on ingest.

Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x2379e348, ext:63767206896, loc:(*time.Location)(nil)}, Meta:{"raw_index":"logs-windows.forwarded-default"}, Fields:{"agent":{"ephemeral_id":"b512edb8-9998-42bd-8941-e1e53d750cc9","hostname":"logcollector2","id":"ef67a54e-002d-4c31-a1c8-931a1a56bea4","name":"logcollector2","type":"filebeat","version":"7.14.1"},"data_stream":{"dataset":"windows.forwarded","namespace":"default","type":"logs"},"ecs":{"version":"1.10.0"},"elastic_agent":{"id":"ef67a54e-002d-4c31-a1c8-931a1a56bea4","snapshot":false,"version":"7.14.1"},"event":{"code":"22","created":"2021-09-14T09:20:46.257Z","dataset":"windows.forwarded","kind":"event","provider":"Microsoft-Windows-Sysmon"},"host":{"name":"DEVICENAME"},"input":{"type":"winlog"},"log":{"level":"information"},"tags":["forwarded"],"winlog":{"api":"wineventlog","channel":"Microsoft-Windows-Sysmon/Operational","computer_name":"DEVICENAME","event_data":{"Image":"\u003cunknown process\u003e","ProcessGuid":"{00000000-0000-0000-0000-000000000000}","ProcessId":"6968","QueryName":"enterpriseregistration.windows.net","QueryResults":"type: 5 adrs.privatelink.msidentity.com;type: 5 www.tm.prd.adrs.akadns.net;::ffff:20.190.129.168;","QueryStatus":"0","RuleName":"-","UtcTime":"2021-09-14 09:01:34.006"},"event_id":"22","process":{"pid":2412,"thread":{"id":3596}},"provider_guid":"{5770385f-c22a-43e0-bf4c-06f5698ffbd9}","provider_name":"Microsoft-Windows-Sysmon","record_id":12337,"user":{"domain":"NT AUTHORITY","identifier":"S-1-5-18","name":"SYSTEM","type":"User"},"version":5}}, Private:(*cursor.updateOp)(0xc0027a8980), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [related.ip] of type [ip] in document with id 'Hz-b43sBuVXDW4lt2EVA'. Preview of field's value: '_ingest._value'","caused_by":{"type":"illegal_argument_exception","reason":"'_ingest._value' is not an IP string literal."}}

As this shows, there is a mapping parser issue which seems related to an IP field not being an IP? I'm not sure if there is anything else I can check or change to try and debug further?

This error is shown in Discover for any events where event.dataset : elastic_agent.filebeat

Thanks

Phil

2

This looks like a bug in the pipeline for sysmon. Will you please open a bug report in https://github.com/elastic/integrations/issues/new.

Looking at the pipeline for the forwarded events vs the pipeline for the sysmon dataset, I think the two got out of sync. The forwarded one needs the {{ }} to make it a templated value. You can patch your copy of the pipeline temporarily by edit it in Kibana and see if that fixes it. That would be under Stack Management -> Ingest Node Pipelines -> logs-windows.forwarded-1.1.3-sysmon_operational -> Edit. Find the last foreach that has the append and add the braces.

Bug:

Correct:

Make it have value: "{{{_ingest._value}}}".

Screen Shot 2021-09-14 at 10.34.38 AM
Screen Shot 2021-09-14 at 10.34.38 AM1114×1022 51.6 KB

3

Thank you for the response. My ingest pipeline is showing as v1.0.0 rather than v1.1.3 so seems I may have other issues. I did try the change you suggested (at a different line) but it's still not working so perhaps my pipeline will update on a future release and I'll test again then.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.