Filebeat Azure module - mapper_parsing_exception for field azure.signinlogs.properties.conditional_access_audiences

Elastic Stack Beats
,
1

Hello

It seems there was a change in the Azure Singin logs syntax since 2025-06-04.
Since that date the amount of logs grabbing by our Filebeat dramatically decreased. Debug log shows it is caused by "azure.signinlogs.properties.conditional_access_audiences" which is missing in the module config.

Below there is a debug log from one of many events:

2025-06-05T10:03:28.483Z	WARN	[elasticsearch]	elasticsearch/client.go:416	Cannot index event (status=400): dropping event! Enable debug logs to view the event and cause.
2025-06-05T10:03:28.484Z	DEBUG	[elasticsearch]	elasticsearch/client.go:417	Cannot index event 
publisher.Event{
	Content:beat.Event{
	  Timestamp:time.Date(2025, time.June, 5, 10, 3, 24, 879225751, time.Local), 
	  Meta:{
	    "pipeline":"filebeat-7.17.28-azure-signinlogs-pipeline"
	  },
	  Fields:{
	    "agent":{
	      "ephemeral_id":"<REDACTED>",
	      "hostname":"<REDACTED>",
	      "id":"<REDACTED>",
	      "name":"<REDACTED>",
	      "type":"filebeat",
	      "version":"7.17.28"
	    },
	    "azure":{
	      "consumer_group":"$Default",
	      "enqueued_time":"2025-06-05T08:44:36.293Z",
	      "eventhub":"<REDACTED>",
	      "offset":<REDACTED>,
	      "sequence_number":<REDACTED>
	    },
	    "ecs":{
	      "version":"1.12.0"
	    },
	    "event":{
	      "dataset":"azure.signinlogs",
	      "module":"azure"
	    },
	    "fileset":{
	      "name":"signinlogs"
	    },
	    "host":{
	      "architecture":"x86_64",
	      "containerized":false,
	      "hostname":"<REDACTED>",
	      "id":"<REDACTED>",
	      "ip":["<REDACTED>"],
	      "mac":["<REDACTED>"],
	      "os":{
	        "codename":"<REDACTED>",
	        "family":"debian",
	        "kernel":"<REDACTED>",
	        "name":"Debian GNU/Linux",
	        "platform":"debian",
	        "type":"linux",
	        "version":"<REDACTED>"
	      }
	    },
	    "input":{
	      "type":"azure-eventhub"
	    },
	    "message":"<REDACTED>",
	    "service":{
	      "type":"azure"
	    },
	    "tags":["forwarded"]
	  }, 
	  Private:[]uint8{
	    <REDACTED>
	  },
	  TimeSeries:false
	}, 
	Flags:0x1, 
	Cache:publisher.EventCache{m:common.MapStr(nil)}
} 
(status=400): {"type":"mapper_parsing_exception","reason":"object mapping for [azure.signinlogs.properties.conditional_access_audiences] tried to parse field [null] as object, but found a concrete value"}, dropping event!

We find out a workaround - put a remove processor to filebeat-7.17.28-azure-signinlogs-pipeline:

  {
    "remove": {
      "field": "azure.signinlogs.properties.conditional_access_audiences"
    }
  }

May I ask to correct this so the field azure.signinlogs.properties.conditional_access_audiences will be correctly processed?

2

Hello,

I just had the same issue with the Azure Elastic Agent Integration, there was no change on Elastic side, it was something on Microsoft side.

The field azure.signinlogs.properties.conditional_access_audiences is derived from a field named condtionalAccessAudiences and Microsoft can send this field sometimes as a list of strings and sometimes as an object, which leads to mapping conflict.

Besides that, the field azure.signinlogs.properties.conditional_access_audiences is not explictly mapped, so it will be dynamically mapped depending on the first value a new index receives.

I'm not sure how to fix this for Filebeat as I do not use it, but I've opened an issue for the Agent Integration with a work around here: [Azure Logs] Mapping conflict an dropped events because of field that can be both a string and an object. · Issue #14178 · elastic/integrations · GitHub