# Filebeat Azure Activity Logs fail to index when field azure.activitylogs.identity.claims is string instead of json

**URL:** <https://discuss.elastic.co/t/filebeat-azure-activity-logs-fail-to-index-when-field-azure-activitylogs-identity-claims-is-string-instead-of-json/268516>\
**Category:** Beats\
**Tags:** beats-module, filebeat\
**Created:** [March 26, 2021, 4:02pm UTC](https://discuss.elastic.co/t/filebeat-azure-activity-logs-fail-to-index-when-field-azure-activitylogs-identity-claims-is-string-instead-of-json/268516 "2021-03-26T16:02:47Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![tobias.berg](https://avatars.discourse-cdn.com/v4/letter/t/ac8455/32.png) [@tobias.berg](https://discuss.elastic.co/u/tobias.berg)\
**Post date:** [March 26, 2021, 4:02pm UTC](https://discuss.elastic.co/t/filebeat-azure-activity-logs-fail-to-index-when-field-azure-activitylogs-identity-claims-is-string-instead-of-json/268516/1 "2021-03-26T16:02:47Z")

</div>

Hi,

We have noticed that sometimes, the Azure Activity logs contains serilaized json instead of pure json in the azure.activitylogs.identity field. Thus, these logs fail to index with a mapping parsing exception.

We have created a workaround by adding an extra JSON processor to the Filebeat Azure Activity Logs pipeline (filebeat-7.11.1-azure-activitylogs-pipeline in our case). I think it would be good if this could be added in the official pipeline. I'm happy to submit a PR if you want me to.

An example document with serialized json in the identity field:

```auto
{
  "agent": {
    "ephemeral_id": "myid",
    "hostname": "myhost",
    "id": "id",
    "name": "filebeat-64b5dc8949-md5p9",
    "type": "filebeat",
    "version": "7.11.1"
  },
  "azure": {
    "consumer_group": "filebeat",
    "enqueued_time": "2021-03-25T09:27:44.332Z",
    "eventhub": "myeventhub",
    "offset": 2147509126360,
    "sequence_number": 2773145
  },
  "cloud": {
    "account": {},
    "instance": {
      "id": "myid",
      "name": "myname"
    },
    "machine": {
      "type": "Standard_E8s_v3"
    },
    "provider": "azure",
    "region": "westeurope"
  },
  "ecs": {
    "version": "1.7.0"
  },
  "event": {
    "dataset": "azure.activitylogs",
    "module": "azure"
  },
  "fileset": {
    "name": "activitylogs"
  },
  "input": {
    "type": "azure-eventhub"
  },
  "message": "{\"Authorization\":\"null\",\"Claims\":\"{\\\"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress\\\":\\\"Microsoft.RecoveryServices\\\"}\",\"DeploymentUnit\":\"myunit\",\"EventId\":162,\"EventName\":\"AzureBackupActivityLog\",\"ResultDescription\":\"Backup Succeeded\",\"category\":\"Administrative\",\"correlationId\":\"666a2ef3-a0a8-4bb3-a977-1d60aa5fb185\",\"durationMs\":0,\"eventName\":\"Backup\",\"identity\":\"{\\\"claims\\\":{\\\"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress\\\":\\\"Microsoft.RecoveryServices\\\"}}\",\"level\":\"Informational\",\"location\":\"westeurope\",\"operationId\":\"bd686d52-0f93-440e-85a7-8ed03e2a8d75\",\"operationName\":\"Microsoft.RecoveryServices/vaults/backupFabrics/protectionContainers/protectedItems/backup/action\",\"operationVersion\":\"null\",\"properties\":{\"Entity Name\":\"mymachine\",\"Job Id\":\"jobid\",\"Start Time\":\"2021-03-24 21:50:40Z\"},\"resourceId\":\"/SUBSCRIPTIONS/111B1A11-11E1-1FDE-1111-11FAD111ECCA/RESOURCEGROUPS/MY-GROUP/PROVIDERS/MICROSOFT.RECOVERYSERVICES/VAULTS/RSV-DATAPROTECTION\",\"resultType\":\"Succeeded\",\"time\":\"2021-03-25T09:22:28.4002017Z\"}",
  "service": {
    "type": "azure"
  },
  "tags": [
    "forwarded"
  ]
}

```

By adding this JSON processor after the first JSON processor in the pipeline, this json gets parsed and the log post gets indexed:

```auto
{
      "json": {
        "field": "azure.activitylogs.identity",
        "if": "ctx.azure?.activitylogs?.identity instanceof String",
        "ignore_failure": true
      }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 23, 2021, 6:03pm UTC](https://discuss.elastic.co/t/filebeat-azure-activity-logs-fail-to-index-when-field-azure-activitylogs-identity-claims-is-string-instead-of-json/268516/2 "2021-04-23T18:03:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.