Elastic detection rules fail

[xqaiviwjxzw]

Rule failure at May 29, 2023 @ 12:09:44.438
Bulk Indexing of signals failed: ResponseError: search_phase_execution_exception

Caused by:

illegal_argument_exception: Can't sort on field [event.ingested]; the field has incompatible sort types: [STRING] and [LONG] across shards!

Elasticity detection rules display a large number of fail errors. What is the reason? What is the solution? Thanks.

[vitaliidm (Vitalii Dmyterko)]

Hi @xqaiviwjxzw

Can you please tell which version of Elasticsearch/Kibana are you using?
Looking at error, it seems like your rule queries indices that have different mapping for "event.ingested". According to ECS schema event.ingested should be of date format: Event Fields | Elastic Common Schema (ECS) Reference [8.8] | Elastic
Can you please check mapping for queried indices to understand whether my hypothesis is correct?
If it so, there are few ways to proceed

1. Ensure all indices that rule queries, have the same mapping for event.ingested. Can be achieved by editing rule, removing indices that have different mapping. Another rule can be added that have the removed indices from the first one
2. Reindexing the data to ensure it has the same format, preferably ECS compliant date across all indices

Thanks, Vitalii