How to remove custom metadata fields from users and roles

Wolfram_Haussig · March 9, 2021, 7:34am

Hi,

I am running a 7.10 Stack with Security enabled and I am currently playing around with the ElasticSearch REST API for creating users, roles and so on. The usecase here is to add advanced information to the security objects(like description, the team it belongs to, last change date, ...).

For this, I use the metadata fields which works fine for adding information but I cannot find a way to remove the fields. Add a custom field to an existing user:

PUT _security/user/testusr
{
  "roles": [
    "reporting_user"
  ],
  "metadata": {
    "description": "abcdef"
  }
}

=> the field metadata.description now contains the value abcdef.

Here is what I tried so far to remove the field:

remove the custom field from the request

PUT _security/user/testusr
{
  "roles": [
    "reporting_user"
  ],
  "metadata": {
  }
}

=> the field metadata.description still contains the value abcdef.
2. set custom field to null

PUT _security/user/testusr
{
  "roles": [
    "reporting_user"
  ],
  "metadata": {
     "description": null
  }
}

=> the field metadata.description still exists - now with a nullvalue.
3. remove metadata element in request

PUT _security/user/testusr
{
  "roles": [
    "reporting_user"
  ]
}

=> the field metadata.description is gone but all other metadata fields too.

How can I remove a metadata field without removing the others?

Is it a feature that not supplying the metadata in the request removes all stored metadata or is it a bug? I would expect it to behave the same as providing an empty metadata element.

Best regards
Wolfram

Albert_Zaharovits · March 10, 2021, 7:46pm

Hi @Wolfram_Haussig ,

The PUT APIs are designed to overwrite the complete entity, which includes the metadata field (unlike the PATCH HTTP verb). There's only a single exception for user entities, and that's the password field.

This means that the complete metadata must be specified on each API, no merges happen server-side.

Wolfram_Haussig · March 10, 2021, 7:53pm

Hello @Albert_Zaharovits ,

Thank you for your response! I am a bit confused though as I already tried providing the metadata field(see my first post with an empty metadata) and the fields still exist after that!

If it would work as you described it would be fine for me though..

Best regards
Wolfram

TimV · March 11, 2021, 7:54am

I think this is a bug specifically for users.

Because we need to preserve a user's password when they are updated, internally PUT /_security/user/{name} will perform an update on the underlying document unless the request body includes a password (or password hash).

The semantics of that update means that metadata fields are not removed, even though they are supposed to be (and are for other object types like roles).

Until we work out how to fix this (with minimal impact to anyone who relies on the existing behaviour) the only workaround I know of is to update the user's password as part of the PUT request.

Wolfram_Haussig · March 11, 2021, 8:47am

Hello @TimV ,

Thank you for your confirmation! Unfortunately, the workaround of including the password will not always work for us because I might want to annotate a user where I do not know the password so I think I will go with setting it to null for now.

Shall I create a github issue for it?

Best regards
Wolfram

TimV · March 11, 2021, 9:17am

No need, I already have.

github.com/elastic/elasticsearch

Put User doesn't always overwrite custom metadata

opened 08:24AM - 11 Mar 21 UTC

tvernum

>bug :Security/Authentication Team:Security

https://discuss.elastic.co/t/how-to-remove-custom-metadata-fields-from-users-and…-roles/266646/2 By design, a PUT on a user does not overwrite their password unless the password (or hash) is in the request body. That means, that within the NativeUsersStore, the Put will actually perform an update on the underlying document. Because `metadata` is stored as a nested object, the semantics of that update means that metadata fields are not always removed from the document, even though they are supposed to be (and are for other object types like roles). To be extra confusing putting a user with `metadata: {}` will preserve all existing metadata, but not specifying `metadata` at all will remove all metadata (and `metadata: null` is not allowed by the rest parser). We will need to think carefully about how to fix this without breaking existing workflows that rely on the bug. Examples below: ``` GET /.security/_doc/user-test {} === { "_index": ".security-7", "_type": "_doc", "_id": "user-test", "found": false } === PUT /_security/user/test {} {"roles":[],"password":"this is a password","metadata":{"test 1":1}} === { "created": true } === GET /.security/_doc/user-test {} === { "_index": ".security-7", "_type": "_doc", "_id": "user-test", "_version": 8, "_seq_no": 23, "_primary_term": 2, "found": true, "_source": { "username": "test", "password": "$2a$10$wDDS11yWheRfK2ypdYnlkOvX5Mbh/h38i9Ig9hE.1QHpUY0RlFeLq", "roles": [], "full_name": null, "email": null, "metadata": { "test 1": 1 }, "enabled": true, "type": "user" } } === GET /_security/user/test {} === { "test": { "username": "test", "roles": [], "full_name": null, "email": null, "metadata": { "test 1": 1 }, "enabled": true } } === PUT /_security/user/test {} {"roles":[],"metadata":{"test 2":2}} === { "created": false } === GET /.security/_doc/user-test {} === { "_index": ".security-7", "_type": "_doc", "_id": "user-test", "_version": 9, "_seq_no": 24, "_primary_term": 2, "found": true, "_source": { "username": "test", "password": "$2a$10$wDDS11yWheRfK2ypdYnlkOvX5Mbh/h38i9Ig9hE.1QHpUY0RlFeLq", "roles": [], "full_name": null, "email": null, "metadata": { "test 1": 1, "test 2": 2 }, "enabled": true, "type": "user" } } === GET /_security/user/test {} === { "test": { "username": "test", "roles": [], "full_name": null, "email": null, "metadata": { "test 2": 2, "test 1": 1 }, "enabled": true } } === PUT /_security/user/test {} {"roles":[],"metadata":{}} === { "created": false } === GET /.security/_doc/user-test {} === { "_index": ".security-7", "_type": "_doc", "_id": "user-test", "_version": 9, "_seq_no": 24, "_primary_term": 2, "found": true, "_source": { "username": "test", "password": "$2a$10$wDDS11yWheRfK2ypdYnlkOvX5Mbh/h38i9Ig9hE.1QHpUY0RlFeLq", "roles": [], "full_name": null, "email": null, "metadata": { "test 1": 1, "test 2": 2 }, "enabled": true, "type": "user" } } === GET /_security/user/test {} === { "test": { "username": "test", "roles": [], "full_name": null, "email": null, "metadata": { "test 2": 2, "test 1": 1 }, "enabled": true } } === PUT /_security/user/test {} {"roles":[]} === { "created": false } === GET /.security/_doc/user-test {} === { "_index": ".security-7", "_type": "_doc", "_id": "user-test", "_version": 10, "_seq_no": 25, "_primary_term": 2, "found": true, "_source": { "username": "test", "password": "$2a$10$wDDS11yWheRfK2ypdYnlkOvX5Mbh/h38i9Ig9hE.1QHpUY0RlFeLq", "roles": [], "full_name": null, "email": null, "metadata": null, "enabled": true, "type": "user" } } === GET /_security/user/test {} === { "test": { "username": "test", "roles": [], "full_name": null, "email": null, "metadata": {}, "enabled": true } } ```

system · April 8, 2021, 9:18am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.