Elastic Security Rule exception

[Billz1026 (Tharaka Bilesh)]

Hi,

I have been using elastic security as a SIEM for my organization for some time. Due to insufficient space, I had deleted one index. Thereafter I saw failures of the detection rules which was not the case before.
Below is one failure message.

The error is as follows.

An error occurred during rule execution: message: "verification_exception: [verification_exception] Reason: Found 5 problems line 2:4: Cannot use field [event.category] due to ambiguities being mapped as [2] incompatible types: [text] in [winlogbeat-7.12.0], [keyword] in [.ds-logs-endpoint.events.file-default-2022.02.03-000001, .ds-logs-endpoint.events.file-default-2022.03.08-000002, .ds-logs-endpoint.events.library-default-2022.02.03-000001, .ds-logs-endpoint.events.library-default-2022.03.08-000002, .ds-logs-endpoint.events.network-default-2022.02.03-000001, .ds-logs-endpoint.events.network-default-2022.03.08-000002, .ds-logs-endpoint.events.process-default-2022.02.03-000001, .ds-logs-endpoint.events.process-default-2022.03.08-000002, .ds-logs-endpoint.events.registry-default-2022.02.03-000001, .ds-logs-endpoint.events.registry-default-2022.03.08-000002, .ds-logs-endpoint.events.security-default-2022.02.03-000001, .ds-logs-endpoint.events.security-default-2022.03.08-000002, .ds-logs-windows.forwarded-default-2022.02.21-000001, .ds-logs-windows.powershell-default-2022.02.03-000001, .ds-logs-windows.powershell-default-2022.03.08-000002, .ds-logs-windows.powershell_operational-default-2022.02.03-000001, .ds-logs-windows.powershell_operational-default-2022.03.08-000002, .ds-logs-windows.sysmon_operational-default-2022.02.03-000001, .ds-logs-windows.sysmon_operational-default-2022.03.08-000002] line 2:15: Cannot use field [event.type] due to ambiguities being mapped as [2] incompatible types: [text] in [winlogbeat-7.12.0], [keyword] in [.ds-logs-endpoint.events.file-default-2022.02.03-000001, .ds-logs-endpoint.events.file-default-2022.03.08-000002, .ds-logs-endpoint.events.library-default-2022.02.03-000001, .ds-logs-endpoint.events.library-default-2022.03.08-000002, .ds-logs-endpoint.events.network-default-2022.02.03-000001, .ds-logs-endpoint.events.network-default-2022.03.08-000002, .ds-logs-endpoint.events.process-default-2022.02.03-000001, .ds-logs-endpoint.events.process-default-2022.03.08-000002, .ds-logs-endpoint.events.registry-default-2022.02.03-000001, .ds-logs-endpoint.events.registry-default-2022.03.08-000002, .ds-logs-endpoint.events.security-default-2022.02.03-000001, .ds-logs-endpoint.events.security-default-2022.03.08-000002, .ds-logs-windows.forwarded-default-2022.02.21-000001, .ds-logs-windows.powershell-default-2022.02.03-000001, .ds-logs-windows.powershell-default-2022.03.08-000002, .ds-logs-windows.powershell_operational-default-2022.02.03-000001, .ds-logs-windows.powershell_operational-default-2022.03.08-000002, .ds-logs-windows.sysmon_operational-default-2022.02.03-000001, .ds-logs-windows.sysmon_operational-default-2022.03.08-000002] line 2:44: Cannot use field [file.extension] due to ambiguities being mapped as [2] incompatible types: [text] in [winlogbeat-7.12.0], [keyword] in [.ds-logs-endpoint.events.file-default-2022.02.03-000001, .ds-logs-endpoint.events.file-default-2022.03.08-000002, .ds-logs-windows.forwarded-default-2022.02.21-000001, .ds-logs-windows.powershell-default-2022.02.03-000001, .ds-logs-windows.powershell-default-2022.03.08-000002, .ds-logs-windows.powershell_operational-default-2022.02.03-000001, .ds-logs-windows.powershell_operational-default-2022.03.08-000002, .ds-logs-windows.sysmon_operational-default-2022.02.03-000001, .ds-logs-windows.sysmon_operational-default-2022.03.08-000002] line 3:7: Cannot use field [process.name] due to ambiguities being mapped as [2] incompatible types: [text] in [winlogbeat-7.12.0], [keyword] in [.ds-logs-endpoint.events.file-default-2022.02.03-000001, .ds-logs-endpoint.events.file-default-2022.03.08-000002, .ds-logs-endpoint.events.library-default-2022.02.03-000001, .ds-logs-endpoint.events.library-default-2022.03.08-000002, .ds-logs-endpoint.events.network-default-2022.02.03-000001, .ds-logs-endpoint.events.network-default-2022.03.08-000002, .ds-logs-endpoint.events.process-default-2022.02.03-000001, .ds-logs-endpoint.events.process-default-2022.03.08-000002, .ds-logs-endpoint.events.registry-default-2022.02.03-000001, .ds-logs-endpoint.events.registry-default-2022.03.08-000002, .ds-logs-endpoint.events.security-default-2022.02.03-000001, .ds-logs-endpoint.events.security-default-2022.03.08-000002, .ds-logs-windows.forwarded-default-2022.02.21-000001, .ds-logs-windows.powershell-default-2022.02.03-000001, .ds-logs-windows.powershell-default-2022.03.08-000002, .ds-logs-windows.powershell_operational-default-2022.02.03-000001, .ds-logs-windows.powershell_operational-default-2022.03.08-000002, .ds-logs-windows.sysmon_operational-default-2022.02.03-000001, .ds-logs-windows.sysmon_operational-default-2022.03.08-000002] line 7:11: Cannot use field [file.name] due to ambiguities being mapped as [2] incompatible types: [text] in [winlogbeat-7.12.0], [keyword] in [.ds-logs-endpoint.events.file-default-2022.02.03-000001, .ds-logs-endpoint.events.file-default-2022.03.08-000002, .ds-logs-endpoint.events.library-default-2022.02.03-000001, .ds-logs-endpoint.events.library-default-2022.03.08-000002, .ds-logs-windows.forwarded-default-2022.02.21-000001, .ds-logs-windows.powershell-default-2022.02.03-000001, .ds-logs-windows.powershell-default-2022.03.08-000002, .ds-logs-windows.powershell_operational-default-2022.02.03-000001, .ds-logs-windows.powershell_operational-default-2022.03.08-000002, .ds-logs-windows.sysmon_operational-default-2022.02.03-000001, .ds-logs-windows.sysmon_operational-default-2022.03.08-000002]" name: "Execution of File Written or Modified by PDF Reader" id: "6a7fd802-2746-11ec-ad7a-d9be8782438c" rule id: "1defdd62-cd8d-426e-a246-81a37751bb2b" signals index: ".siem-signals-default"```

From the error I can see, the detection is failing due to two different types of event.category field in different indexes. When I checked the index patterns, I could verify two different types of the event.category field. But I couldn't edit the type of the filed.

Hence I would really appreciate if someone can give a hint to resolve this.

BR,
Billz

[Mike_Paquette]

Hi @Billz1026, thanks for the post.

The error message tells us that there are mapping problems with your winlogbeat-7.12.0 index/indices.

The message reports that event.category, event.type, file.extension, process.name, and file.name are being indexed as data type text, which is not compatible with the required ECS-defined data type of keyword.

One hypothesis is that when you deleted one index, you somehow removed /altered the index mapping template(s) for the winlogbeat-7.12.0 index, and now new events being written to that index are no longer compliant with ECS.

If this hypothesis turns out to be true, you will need to update the index mapping templates for your winlogbeat-7.12.0 index pattern, and then as new data is correctly indexed, the rule failures should stop.

Additionally, you may choose to go back and correct the improperly mapped events via some kind of re-index operation.

Could you share your index mappings for your winlogbeat-7.12.0 index pattern so we can see if this is the problem? (Stack Management -> Index Management -> select winlogbeat-7.12.0 then select Mappings)