# Filebeat o365 module - Parsing o365.audit.data field

**URL:** <https://discuss.elastic.co/t/filebeat-o365-module-parsing-o365-audit-data-field/261591>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 19, 2021, 8:47pm UTC](https://discuss.elastic.co/t/filebeat-o365-module-parsing-o365-audit-data-field/261591 "2021-01-19T20:47:28Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![opiedrah](https://avatars.discourse-cdn.com/v4/letter/o/5f9b8f/32.png) [@opiedrah](https://discuss.elastic.co/u/opiedrah)\
**Post date:** [January 19, 2021, 8:47pm UTC](https://discuss.elastic.co/t/filebeat-o365-module-parsing-o365-audit-data-field/261591/1 "2021-01-19T20:47:28Z")

</div>

Hi,  
I have filebeat 7.10 running with module o365 beat. Similar to how extendedproperties filed for azure active directory workload is parsed. How can i parse this o365.audit.data filed.

the value in this field looks like this:

```auto
{"Version":"3.0","VendorName":"Microsoft","ProviderName":"OATP","AlertType":"ea8169fa-0678-4751-8854-aebea7adeceb","StartTimeUtc":"2021-01-19T19:37:54Z","EndTimeUtc":"2021-01-19T19:37:54Z","TimeGenerated":"2021-01-19T20:09:05.023Z","ProcessingEndTime":"2021-01-19T20:17:59.467675Z","Status":"InProgress","Severity":"Informational","ConfidenceLevel":"Unknown","ConfidenceScore":1.0,"IsIncident":false,"ProviderAlertId":"4827709d-0d0e-f576-d600-08d8bcb196e8","SystemAlertId":null,"CorrelationKey":"9762deaa-51ba-4218-af64-ae1932dd8c42","InvestigationIds":["urn:ZappedPhishInvestigation:84ffdeeb9aecd7432b151670ff5ce101"],"Intent":"Probing","ResourceIdentifiers":[{"$id":"1","AadTenantId":"caece813-0c48-4923-8c62-8676bc93605f","Type":"AAD"}],"AzureResourceId":null,"WorkspaceId":null,"WorkspaceSubscriptionId":null,"WorkspaceResourceGroup":null,"AgentId":null,"AlertDisplayName":"Email messages containing phish URLs removed after delivery","Description":"Emails with phish URLs that were delivered and later removed -V1.0.0.5","ExtendedLinks":[{"Href":"https://protection.office.com/viewalerts?id=4827709d-0d0e-f576-d600-08d8bcb196e8","Category":null,"Label":"alert","Type":"webLink"}],"Metadata":{"CustomApps":null,"GenericInfo":null},"Entities":[{"$id":"2","Url":"http://james.boiledpro.com/#amFtZXNAY29hY2hlbGxhLmNvbQ==","Type":"url","ClickCount":0,"EmailCount":1,"Urn":"urn:UrlEntity:2f1b33cf86b82406658d447bbd5bb64a","Source":"OATP","FirstSeen":"2021-01-19T20:12:45"},{"$id":"3","Files":[{"$id":"4","Name":"logo.png","FileHashes":[{"$id":"5","Algorithm":"SHA256","Value":"3B41C7D55985B85B4761BDEEF79CE73207722C0199D7036A337290A01EC26EAC","Type":"filehash"}],"Type":"file","MalwareFamily":""}],"Recipient":"james@contosco.com","Urls":["http://james.boiledpro.com/#amFtZXNAY29hY2hlbGxhLmNvbQ=="],"Threats":["ZapPhish"],"Sender":"support@tap4stuff.com","P1Sender":"010101771bf4fae3-a2418c54-a719-4f81-ad13-f04cf68dd85c-000000@us-west-2.amazonses.com","P1SenderDomain":"us-west-2.amazonses.com","SenderIP":"54.240.27.192","P2Sender":"support@tap4stuff.com","P2SenderDisplayName":"contosco Helpdesk","P2SenderDomain":"tap4stuff.com","ReceivedDate":"2021-01-19T18:42:58","NetworkMessageId":"ccabaffb-fad9-4f2d-b8a4-08d8bcaa09fe","InternetMessageId":"<010101771bf4fae3-a2418c54-a719-4f81-ad13-f04cf68dd85c-000000@us-west-2.amazonses.com>","Subject":"ACTION REQUIRED: New Yearly Update For james@contosco.com","AntispamDirection":"Inbound","DeliveryAction":"Delivered","Language":"en","DeliveryLocation":"Inbox","Type":"mailMessage","Urn":"urn:MailEntity:4bcea468d90fadc679fefb0cf6e6ea93","Source":"OATP","FirstSeen":"2021-01-19T20:12:45"},{"$id":"6","MailboxPrimaryAddress":"james@contosco.com","Upn":"james@contosco.com","RiskLevel":"None","Type":"mailbox","Urn":"urn:UserEntity:775ee0a770248e2050fb52725fc3d615","Source":"OATP","FirstSeen":"2021-01-19T20:12:45"}],"LogCreationTime":"2021-01-19T20:17:59.467675Z","MachineName":"SN1NAM02BG401","SourceTemplateType":"Threat_Single","Category":"ThreatManagement"}

```

Or like this

```auto
o365.audit.data: {"ts":"2021-01-19 04:14:52Z","te":"2021-01-19 04:14:52Z","an":"Custom: Risky Activity from Foreign Countries","ad":"Activity policy 'Custom: Risky Activity from Foreign Countries' was triggered by 'First Last(first.last@contosco.com)'","f3u":"first.last@contosco.com","alk":"https://contosco.portal.cloudappsecurity.com/#/alerts/60065cc0237fdbb8908af067","plk":"https://contosco.portal.cloudappsecurity.com/#/policy/?id=eq(5b076e0df82b1b6d8dfa9f9a,)","mat":"MCAS_ALERT_CABINET_EVENT_MATCH_AUDIT"}

```

Looking at the original module author

> **[counteractive/o365beat](https://github.com/counteractive/o365beat)**
>
> Elastic Beat for fetching and shipping Office 365 audit events

  
in the section under FAQ that reads, how elastic has integrated this module into filebeat is very different in how the original author performed this activity, i don't know how to apply this script to filebeat to parse this filed correctly.

I would appreciate any guidance on how to parse o365.audt.data filed.

Notes from the module author from its original version:

**Can I parse event fields like `ExtendedProperties` and `Parameters` that contain arrays of name-value pairs on the client side before shipping them?**

As of version 1.5.1, the beat imports the [`script` processor](https://www.elastic.co/guide/en/beats/filebeat/current/processor-script.html) and provides a sample processor script in `o365beat.reference.yml` to convert fields that contain arrays of name-value pairs into a "normal" object. See [this issue](https://github.com/counteractive/o365beat/issues/41) for more discussion.

now the script he is referring to is from [https://github.com/counteractive/o365beat/blob/master/o365beat.reference.yml](https://github.com/counteractive/o365beat/blob/master/o365beat.reference.yml)

```auto
 # - script:
  # when:
  # or:
  # - has_fields: ['Parameters']
  # - has_fields: ['ExtendedProperties']
  # lang: javascript
  # id: name_value_array_parser
  # source: >
  # function process(event){
  # var processed = event.Get('processed') || {};
  # var parameters = event.Get('Parameters')
  # if(!!parameters && !!parameters.length){
  # processed.Parameters = processed.Parameters || {};
  # for(var i = 0; i < parameters.length; i++){
  # var p = parameters[i];
  # if(p.Name) processed.Parameters[p.Name] = p.Value;
  # }
  # }
  # var extendedProperties = event.Get('ExtendedProperties')
  # if(!!extendedProperties && !!extendedProperties.length){
  # processed.ExtendedProperties = processed.ExtendedProperties || {};
  # for(var i = 0; i < extendedProperties.length; i++){
  # var p = extendedProperties[i];
  # if(p.Name) processed.ExtendedProperties[p.Name] = p.Value;
  # }
  # }
  # event.Put('processed', processed);
  # }

```

would really appreciate any help on this.

---

<div class="post-metadata">

**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\
**Post date:** [January 20, 2021, 9:17am UTC](https://discuss.elastic.co/t/filebeat-o365-module-parsing-o365-audit-data-field/261591/2 "2021-01-20T09:17:26Z")

</div>

I think that using a script processor for this fields is not a bad idea. You can give a try to JSON processor (ingest node): [https://www.elastic.co/guide/en/elasticsearch/reference/current/json-processor.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/json-processor.html)

---

<div class="post-metadata">

**Author:** ![opiedrah](https://avatars.discourse-cdn.com/v4/letter/o/5f9b8f/32.png) [@opiedrah](https://discuss.elastic.co/u/opiedrah)\
**Post date:** [January 21, 2021, 6:56pm UTC](https://discuss.elastic.co/t/filebeat-o365-module-parsing-o365-audit-data-field/261591/3 "2021-01-21T18:56:20Z")

</div>

Thanks Marcin, i am looking to leverage the script processor as documented in the original o365 beat, the JSON processor wouldn't work for some of the other logs that get stored in the o365.audit.data field. I have no idea how to go about it with the filebeat.

Thank you

---

<div class="post-metadata">

**Author:** ![opiedrah](https://avatars.discourse-cdn.com/v4/letter/o/5f9b8f/32.png) [@opiedrah](https://discuss.elastic.co/u/opiedrah)\
**Post date:** [February 2, 2021, 5:59pm UTC](https://discuss.elastic.co/t/filebeat-o365-module-parsing-o365-audit-data-field/261591/4 "2021-02-02T17:59:47Z")

</div>

Anyone have any ideas how i can parse the o365.audit.data fiiled in filebeat?

---

<div class="post-metadata">

**Author:** ![opiedrah](https://avatars.discourse-cdn.com/v4/letter/o/5f9b8f/32.png) [@opiedrah](https://discuss.elastic.co/u/opiedrah)\
**Post date:** [February 3, 2021, 12:24pm UTC](https://discuss.elastic.co/t/filebeat-o365-module-parsing-o365-audit-data-field/261591/5 "2021-02-03T12:24:52Z")

</div>

@jamie.hynds  
Hi Jamie , do you have any insights on this by any chance or know someone that might thanks .

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 3, 2021, 2:25pm UTC](https://discuss.elastic.co/t/filebeat-o365-module-parsing-o365-audit-data-field/261591/6 "2021-03-03T14:25:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.