Mapping conflict

Elastic Search
1

I'm running v8.17.3 Enterprise and mapping conflicts have appeared on these four indexes.

.ds-logs-system.security-default-2025.03.05-001023
.ds-logs-system.security-default-2025.03.06-001025
.ds-logs-system.security-default-2025.03.08-001026
.ds-logs-system.security-default-2025.03.10-001029

I've included a pic of the fields and types that are the source of the mapping conflict.

I've never encountered a mapping conflict before and I don't know why this has occurred now. I'm not positive but the timing is very close to when the cluster was upgraded to 8.17.3. I am not sure what the best way to resolve this issue is so any guidance is appreciated.

Screenshot 2025-03-12 120307
Screenshot 2025-03-12 1203071710×569 44.6 KB

2

You need to upgrade the System and Windows integration, this is a bug caused by this and reverted here.

3

Thank you Leandro. I had updated those integrations to the latest versions prior to posting my message. Do the updated integrations fix the problem going forward? I use the affected fields quite a bit to hunt in Kibana and I can't use them while the mapping conflict exists. Is there a way to fix my existing indices with the mapping conflict or do I need to wait until the affected indices age out and are deleted?

4

What are the current version of the integrations that you have?

The latest version are 1.67.3 for the System integration and 2.5.2 for the Windows integration, check if all your policies are using these latest versions.

I'm not sure if updateding the integration would force a rollover of the datastreams, a rollover is required to create a new backing index with the correct mapping.

If a rollover was not triggered by the upgrade, then you need to force it.

To force a rollover you can make a request in Dev Tools, something like this:

POST logs-system.security-default/_rollover

After that the correct mapping should be applied and work for new documents, the data in older indices will not be fixed, if you need to fix it you will need to reindex the data on the backing indices.

Something like this, I think:

POST _reindex
{
  "conflicts": "proceed",
  "source": {
    "index": ".ds-logs-system.security-default-2025.03.05-001023"
  },
  "dest": {
    "index": "logs-system.security-default",
    "op_type": "create"
  }
}

Or if you do not care/do need the old data, you can just remove the old backing indices.

5

Yes, I updated to those versions of the integrations yesterday. Thanks for your help. I've done a rollover on that index and am now in the process of reindexing the 4 indexes affected. They are all 50+GB each so that will take some time. Hopefully when they are done, the red triangle with the exclamation point will be gone for those fields in Kibana. I appreciate your guidance.