# Ingesting O365 logs but having issues with specific mappings

**URL:** <https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081>\
**Category:** Elasticsearch\
**Created:** [November 20, 2020, 8:04am UTC](https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081 "2020-11-20T08:04:35Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [November 20, 2020, 8:04am UTC](https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081/1 "2020-11-20T08:04:35Z")

</div>

Hi,

We are using Filebeat with the O365 module. These events get shipped to Redis which then Logstash fetches from. The logs are getting ingested but some of the events are having mapping issues with the field "o365.audit.Parameters".

We get these errors from logstash:

```
[2020-11-20T07:37:06,871][WARN][logstash.outputs.elasticsearch][main][output_elasticsearch_redis] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"40f66e6a-7b49-425c-b362-6ace7be3bb63", :_index=>"office365-write", :routi
ng=>nil, :_type=>"_doc", :pipeline=>"office365-geoip"}, #<LogStash::Event:0x1e0feeab>], :response=>{"index"=>{"_index"=>"office365-2020.11-00001", "_type"=>"_doc", "_id"=>"40f66e6a-7b49-425c-b362-6ace7be3bb63", "status"=>400, "error"=>{"type"=>"mapper_pa
rsing_exception", "reason"=>"object mapping for [o365.audit.Parameters] tried to parse field [Parameters] as object, but found a concrete value"}}}}

```

We have the following mapping for the index which we got from exporting it from Filebeat:

GET \_template/office365

```
{
  "office365" : {
    "order" : 100,
    "index_patterns" : [
      "office365-*"
    ],
    "settings" : {
      "index" : {
        "number_of_replicas" : "0",
        "mapping" : {
          "total_fields" : {
            "limit" : "10000"
          }
        }
      }
    },
    "mappings" : {
  "properties" : {
    "o365" : {
      "properties" : {
        "audit" : {
          "properties" : {
            "GroupName" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "ItemType" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "TargetUserOrGroupName" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "UserKey" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "ImplicitShare" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "AlertEntityId" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "EventData" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "Name" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "IntraSystemId" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "Item" : {
              "properties" : {
                "*" : {
                  "type" : "object",
                  "properties" : {
                    "*" : {
                      "type" : "object"
                    }
                  }
                }
              }
            },
            "OriginatingServer" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "Version" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "WebId" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "ClientAppId" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "MailboxOwnerUPN" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "SharePointMetaData" : {
              "properties" : {
                "*" : {
                  "type" : "object"
                }
              }
            },
            "CorrelationId" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "SessionId" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "MailboxOwnerMasterAccountSid" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "UniqueSharingId" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "Status" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            },
            "Parameters" : {
              "properties" : {
                "*" : {
                  "type" : "object",
                  "enabled" : false
                }
              }
            }
          }
        }
      }
    }

```

However it still complains about mapping errors.  
What can we do to resolve this in the best way?

Thank you

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 23, 2020, 12:13am UTC](https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081/2 "2020-11-23T00:13:20Z")

</div>

What it's saying is that it expects an object in the `o365.audit.Parameters` field, as that is what is mapped. But it's only receiving a single item, which it can't then place into an object.

You'd need to look at the event in question a little closer, if you can, and figure out what it's missing.

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [November 27, 2020, 12:16pm UTC](https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081/3 "2020-11-27T12:16:05Z")

</div>

Hello @warkolm @victor.nilsson

We currently have a very similar issue with our o365 ingestion. (7.9.2)

` Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"mapper_parsing_exception","reason":"object mapping for [o365.audit.Parameters] tried to parse field [Parameters] as object, but found a concrete value"}`

Indexation of all o365 logs is failing. This smells like a critical bug to me. Further investigation required..

Grtz

Willem

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [November 27, 2020, 1:51pm UTC](https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081/4 "2020-11-27T13:51:47Z")

</div>

Hello,

Not 100 % sure yet if it's a permanent fix, but for now I got ingestion working again by dropping the problematic field in filebeat.yml:

```
- drop_fields:
    when:
      equals:
        event.module: 'o365'
    fields: ["o365.audit.Parameters"]

```

Grtz

Willem

---

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [November 27, 2020, 2:25pm UTC](https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081/5 "2020-11-27T14:25:15Z")

</div>

We're doing exactly the same. This has to be a bug with the default mapping installed by Filebeat

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [November 27, 2020, 8:33pm UTC](https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081/6 "2020-11-27T20:33:52Z")

</div>

Created [https://github.com/elastic/beats/issues/22780](https://github.com/elastic/beats/issues/22780), feel free to add more information.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 25, 2020, 8:33pm UTC](https://discuss.elastic.co/t/ingesting-o365-logs-but-having-issues-with-specific-mappings/256081/7 "2020-12-25T20:33:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.