# How to remove custom metadata fields from users and roles

**URL:** <https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [March 9, 2021, 7:34am UTC](https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646 "2021-03-09T07:34:09Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [March 9, 2021, 7:34am UTC](https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646/1 "2021-03-09T07:34:09Z")

</div>

Hi,

I am running a 7.10 Stack with Security enabled and I am currently playing around with the ElasticSearch REST API for creating users, roles and so on. The usecase here is to add advanced information to the security objects(like description, the team it belongs to, last change date, ...).

For this, I use the metadata fields which works fine for adding information but I cannot find a way to remove the fields. Add a custom field to an existing user:

```auto
PUT _security/user/testusr
{
  "roles": [
    "reporting_user"
  ],
  "metadata": {
    "description": "abcdef"
  }
}

```

=\> the field `metadata.description` now contains the value `abcdef`.

Here is what I tried so far to remove the field:

1. remove the custom field from the request

```auto
PUT _security/user/testusr
{
  "roles": [
    "reporting_user"
  ],
  "metadata": {
  }
}

```

=\> the field `metadata.description` still contains the value `abcdef`.  
2. set custom field to null

```auto
PUT _security/user/testusr
{
  "roles": [
    "reporting_user"
  ],
  "metadata": {
     "description": null
  }
}

```

=\> the field `metadata.description` still exists - now with a `null`value.  
3. remove metadata element in request

```auto
PUT _security/user/testusr
{
  "roles": [
    "reporting_user"
  ]
}

```

=\> the field `metadata.description` is gone but all other metadata fields too.

How can I remove a metadata field without removing the others?

Is it a feature that not supplying the metadata in the request removes all stored metadata or is it a bug? I would expect it to behave the same as providing an empty metadata element.

Best regards  
Wolfram

---

<div class="post-metadata">

**Author:** ![Albert\_Zaharovits](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/albert_zaharovits/32/24390_2.png) [@Albert\_Zaharovits](https://discuss.elastic.co/u/Albert_Zaharovits)\
**Post date:** [March 10, 2021, 7:46pm UTC](https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646/2 "2021-03-10T19:46:12Z")

</div>

Hi @Wolfram_Haussig ,

The PUT APIs are designed to overwrite the complete entity, which includes the `metadata` field (unlike the PATCH HTTP verb). There's only a single exception for user entities, and that's the `password` field.

This means that the complete metadata must be specified on each API, no merges happen server-side.

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [March 10, 2021, 7:53pm UTC](https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646/3 "2021-03-10T19:53:43Z")

</div>

Hello @Albert_Zaharovits ,

Thank you for your response! I am a bit confused though as I already tried providing the metadata field(see my first post with an empty metadata) and the fields still exist after that!

If it would work as you described it would be fine for me though..

Best regards  
Wolfram

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [March 11, 2021, 7:54am UTC](https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646/4 "2021-03-11T07:54:31Z")

</div>

I think this is a bug specifically for users.

Because we need to preserve a user's password when they are updated, internally `PUT /_security/user/{name}` will perform an update on the underlying document unless the request body includes a password (or password hash).

The semantics of that update means that metadata fields are not removed, even though they are supposed to be (and are for other object types like roles).

Until we work out how to fix this (with minimal impact to anyone who relies on the existing behaviour) the only workaround I know of is to update the user's password as part of the `PUT` request.

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [March 11, 2021, 8:47am UTC](https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646/5 "2021-03-11T08:47:37Z")

</div>

Hello @TimV ,

Thank you for your confirmation! Unfortunately, the workaround of including the password will not always work for us because I might want to annotate a user where I do not know the password so I think I will go with setting it to null for now.

Shall I create a github issue for it?

Best regards  
Wolfram

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [March 11, 2021, 9:17am UTC](https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646/6 "2021-03-11T09:17:35Z")

</div>

> [@Wolfram\_Haussig](#):
>
> Shall I create a github issue for it?

No need, I already have.

> <https://github.com/elastic/elasticsearch/issues/70295>
>
> https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and…-roles/266646/2
> 
> By design, a PUT on a user does not overwrite their password unless the password (or hash) is in the request body.  
> That means, that within the NativeUsersStore, the Put will actually perform an update on the underlying document.
> 
> Because \`metadata\` is stored as a nested object, the semantics of that update means that metadata fields are not always removed from the document, even though they are supposed to be (and are for other object types like roles).
> 
> To be extra confusing putting a user with \`metadata: {}\` will preserve all existing metadata, but not specifying \`metadata\` at all will remove all metadata (and \`metadata: null\` is not allowed by the rest parser).
> 
> We will need to think carefully about how to fix this without breaking existing workflows that rely on the bug. 
> 
> Examples below:
> \`\`\`
> GET /.security/\_doc/user-test {}
> === 
> {
> "\_index": ".security-7",
> "\_type": "\_doc",
> "\_id": "user-test",
> "found": false
> }
> === 
> 
> PUT /\_security/user/test {}
> {"roles":\[\],"password":"this is a password","metadata":{"test 1":1}}
> === 
> { "created": true }
> === 
> 
> GET /.security/\_doc/user-test {}
> === 
> {
> "\_index": ".security-7",
> "\_type": "\_doc",
> "\_id": "user-test",
> "\_version": 8,
> "\_seq\_no": 23,
> "\_primary\_term": 2,
> "found": true,
> "\_source": {
> "username": "test",
> "password": "$2a$10$wDDS11yWheRfK2ypdYnlkOvX5Mbh/h38i9Ig9hE.1QHpUY0RlFeLq",
> "roles": \[\],
> "full\_name": null,
> "email": null,
> "metadata": { "test 1": 1 },
> "enabled": true,
> "type": "user"
> }
> }
> === 
> 
> GET /\_security/user/test {}
> === 
> {
> "test": {
> "username": "test",
> "roles": \[\],
> "full\_name": null,
> "email": null,
> "metadata": { "test 1": 1 },
> "enabled": true
> }
> }
> === 
> 
> PUT /\_security/user/test {}
> {"roles":\[\],"metadata":{"test 2":2}}
> === 
> { "created": false }
> === 
> 
> GET /.security/\_doc/user-test {}
> === 
> {
> "\_index": ".security-7",
> "\_type": "\_doc",
> "\_id": "user-test",
> "\_version": 9,
> "\_seq\_no": 24,
> "\_primary\_term": 2,
> "found": true,
> "\_source": {
> "username": "test",
> "password": "$2a$10$wDDS11yWheRfK2ypdYnlkOvX5Mbh/h38i9Ig9hE.1QHpUY0RlFeLq",
> "roles": \[\],
> "full\_name": null,
> "email": null,
> "metadata": {
> "test 1": 1,
> "test 2": 2
> },
> "enabled": true,
> "type": "user"
> }
> }
> === 
> 
> GET /\_security/user/test {}
> === 
> {
> "test": {
> "username": "test",
> "roles": \[\],
> "full\_name": null,
> "email": null,
> "metadata": {
> "test 2": 2,
> "test 1": 1
> },
> "enabled": true
> }
> }
> === 
> 
> PUT /\_security/user/test {}
> {"roles":\[\],"metadata":{}}
> === 
> { "created": false }
> === 
> 
> GET /.security/\_doc/user-test {}
> === 
> {
> "\_index": ".security-7",
> "\_type": "\_doc",
> "\_id": "user-test",
> "\_version": 9,
> "\_seq\_no": 24,
> "\_primary\_term": 2,
> "found": true,
> "\_source": {
> "username": "test",
> "password": "$2a$10$wDDS11yWheRfK2ypdYnlkOvX5Mbh/h38i9Ig9hE.1QHpUY0RlFeLq",
> "roles": \[\],
> "full\_name": null,
> "email": null,
> "metadata": {
> "test 1": 1,
> "test 2": 2
> },
> "enabled": true,
> "type": "user"
> }
> }
> === 
> 
> GET /\_security/user/test {}
> === 
> {
> "test": {
> "username": "test",
> "roles": \[\],
> "full\_name": null,
> "email": null,
> "metadata": {
> "test 2": 2,
> "test 1": 1
> },
> "enabled": true
> }
> }
> === 
> 
> PUT /\_security/user/test {}
> {"roles":\[\]}
> === 
> { "created": false }
> === 
> 
> GET /.security/\_doc/user-test {}
> === 
> {
> "\_index": ".security-7",
> "\_type": "\_doc",
> "\_id": "user-test",
> "\_version": 10,
> "\_seq\_no": 25,
> "\_primary\_term": 2,
> "found": true,
> "\_source": {
> "username": "test",
> "password": "$2a$10$wDDS11yWheRfK2ypdYnlkOvX5Mbh/h38i9Ig9hE.1QHpUY0RlFeLq",
> "roles": \[\],
> "full\_name": null,
> "email": null,
> "metadata": null,
> "enabled": true,
> "type": "user"
> }
> }
> 
> === 
> GET /\_security/user/test {}
> === 
> {
> "test": {
> "username": "test",
> "roles": \[\],
> "full\_name": null,
> "email": null,
> "metadata": {},
> "enabled": true
> }
> }
> \`\`\`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 8, 2021, 9:18am UTC](https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and-roles/266646/7 "2021-04-08T09:18:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.