# Mapper\_parsing\_exception, hashicorp vault audit, "error" in inner json **URL:** \ **Category:** Beats\ **Tags:** docker, filebeat\ **Created:** [January 21, 2019, 8:51pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-hashicorp-vault-audit-error-in-inner-json/165119 "2019-01-21T20:51:27Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![mkohns](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@mkohns](https://discuss.elastic.co/u/mkohns)\ **Post date:** [January 21, 2019, 8:51pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-hashicorp-vault-audit-error-in-inner-json/165119/1 "2019-01-21T20:51:27Z")
Hi! Following Setup: -\> Docker 18.09 -\> Filebeat 6.4.1 running in container with docker sock mounted -\> Hashicorp Vault running in container with audit enabled -\> extract from filebeat.yml ``` ... filebeat.autodiscover: providers: - type: docker templates: - condition: contains.docker.container.name: "nginx-web" config: - module: nginx access: input: type: docker containers.ids: - "${data.docker.container.id}" - condition: not.contains.docker.container.name: "nginx-web" config: - type: docker containers.ids: - "${data.docker.container.id}" processors: - decode_json_fields: fields: ["message"] target: "" overwrite_keys: true ... ``` vault container writes following line on stdout (hashes are manipulated) ``` { "time": "2019-01-21T18:53:38.322674515Z", "type": "response", "auth": { "client_token": "hmac-sha256:8d8185a5ec2e78206e4fasdfa8b648768c408a7faf85e52c5cf0248f", "accessor": "hmac-sha256:a8c7d8a1a951adfa7c65ef71870ef1f5a494d4330d41cbbf6a", "display_name": "root", "policies": [ "root" ], "token_policies": [ "root" ], "metadata": null, "entity_id": "" }, "request": { "id": "3cdaf04d-b1c2-47a2-a7ca-9309379795bc", "operation": "update", "client_token": "hmac-sha256:8d8185a5ec2e7afd648768c408a7faf85e52c5cf0248f", "client_token_accessor": "hmac-sha256:a8c7d8a1aadfc65ef71870ef1f5a494d4330d41cbbf6a", "path": "sys/audit/file", "data": { "description": "hmac-sha256:1b079dfadf218abd5bab7f895ba073c55118c094e967d4", "local": false, "options": { "file_path": "hmac-sha256:b25edc3c72aadfd8d8909c7f65b99dd75bad5238fe3943dd53" }, "type": "hmac-sha256:56bedf4adf1ec097f7b8da5c29d08db9749d86099d576aa1e" }, "policy_override": false, "remote_address": "172.24.0.1", "wrap_ttl": 0, "headers": {} }, "response": {}, "error": "" } ``` filebeat harvesting this log is not happy `2019-01-21T19:01:19.935Z WARN elasticsearch/client.go:520 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x2018b7b1, ext:63683694075, loc:(*time.Location)(nil)}, Meta:common.MapStr(nil), Fields:common.MapStr{"time":"2019-01-21T19:01:15.538248103Z", "type":"request", "request":map[string]interface {}{"headers":map[string]interface {}{}, "operation":"delete", "client_token":"hmac-sha256:8d8185a5ec2e78206e459a2edb22b15158b648768c408a7faf85e52c5cf0248f", "remote_address":"172.24.0.1", "wrap_ttl":0, "policy_override":false, "id":"8dd93fe4-6485-9e85-1cae-dcbfda4320e5", "client_token_accessor":"hmac-sha256:a8c7d8a1a9518271bea0e9e63b76277c65ef71870ef1f5a494d4330d41cbbf6a", "path":"sys/audit/file", "data":interface {}(nil)}, "source":"/var/lib/docker/containers/65f184d938d27867f7f64e6045b338f377722260a66344d800c8a700b60f042f/65f184d938d27867f7f64e6045b338f377722260a66344d800c8a700b60f042f-json.log", "offset":3209769, "input":common.MapStr{"type":"docker"}, "docker":common.MapStr{"container":common.MapStr{"name":"vault", "image":"vault:0.11.1", "labels":common.MapStr{}, "id":"65f184d938d27867f7f64e6045b338f377722260a66344d800c8a700b60f042f"}}, "host":common.MapStr{"name":"static-1-vm4"}, "stream":"stdout", "prospector":common.MapStr{"type":"docker"}, "message":"{\"time\":\"2019-01-21T19:01:15.538248103Z\",\"type\":\"request\",\"auth\":{\"client_token\":\"hmac-sha256:8d8185a5ec2e78206e459a2edb22b15158b648768c408a7faf85e52c5cf0248f\",\"accessor\":\"hmac-sha256:a8c7d8a1a9518271bea0e9e63b76277c65ef71870ef1f5a494d4330d41cbbf6a\",\"display_name\":\"root\",\"policies\":[\"root\"],\"token_policies\":[\"root\"],\"metadata\":null,\"entity_id\":\"\"},\"request\":{\"id\":\"8dd93fe4-6485-9e85-1cae-dcbfda4320e5\",\"operation\":\"delete\",\"client_token\":\"hmac-sha256:8d8185a5ec2e78206e459a2edb22b15158b648768c408a7faf85e52c5cf0248f\",\"client_token_accessor\":\"hmac-sha256:a8c7d8a1a9518271bea0e9e63b76277c65ef71870ef1f5a494d4330d41cbbf6a\",\"path\":\"sys/audit/file\",\"data\":null,\"policy_override\":false,\"remote_address\":\"172.24.0.1\",\"wrap_ttl\":0,\"headers\":{}},\"error\":\"\"}", "beat":common.MapStr{"name":"static-1-vm4", "hostname":"static-1-vm4", "version":"6.4.1"}, "auth":map[string]interface {}{"metadata":interface {}(nil), "entity_id":"", "client_token":"hmac-sha256:8d8185a5ec2e78206e459a2edb22b15158b648768c408a7faf85e52c5cf0248f", "accessor":"hmac-sha256:a8c7d8a1a9518271bea0e9e63b76277c65ef71870ef1f5a494d4330d41cbbf6a", "display_name":"root", "policies":[]interface {}{"root"}, "token_policies":[]interface {}{"root"}}, "error":""}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc420460dd0), Source:"/var/lib/docker/containers/65f184d938d27867f7f64e6045b338f377722260a66344d800c8a700b60f042f/65f184d938d27867f7f64e6045b338f377722260a66344d800c8a700b60f042f-json.log", Offset:3210665, Timestamp:time.Time{wall:0xbf0925b9583ae8b0, ext:6217344875309200, loc:(*time.Location)(0x1f61700)}, TTL:-1, Type:"docker", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0x200be, Device:0x821}}}, Flags:0x1} (status=400): {"type":"mapper_parsing_exception","reason":"object mapping for [error] tried to parse field [error] as object, but found a concrete value"}` For clarification: In the docker log json from the vault container is a message. This message is also json (inner-json). This json contains a field "error" This field causes the parsing error. **type: mapper\_parsing\_exception** **reason: object mapping for [error] tried to parse field [error] as object, but found a concrete value** I assume .. but really only assume, that this clashes with the "error" array in fields.yml coming from beats. What can I do to make vault audit logging work with filebeat? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 22, 2019, 3:30pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-hashicorp-vault-audit-error-in-inner-json/165119/2 "2019-01-22T15:30:47Z")
Due to `overwrite_keys` you have a clash between the filebeat internal event schema and the json document you are reading. This overwrites/sets the `error` field, which is incompatible to the schema used by beats. First I'd not configure a global processor, as this will also try tp process the nginx modules events. Rather add the `processors` to the local not-nginx configs. Setting the `target` field do a named value (maybe depending on container meta-data) will put the json into a separate namespace, reducing the chance of schema-clashes. You can also post-process your json by using the drop\_fields or rename processors. Full schema docs: [https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields.html](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields.html). The [Beats fields](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-beat.html) define the error object. If you find 'error' to be a string, you could rename it to `error.message`. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 19, 2019, 3:41pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-hashicorp-vault-audit-error-in-inner-json/165119/3 "2019-02-19T15:41:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.