Elasticsearch custom realm, support for wildcard properties

[patrikjetzer]

I have implemented a custom real plugin based on the _x-pack_qa_security-example-spi-extension project. The problem i'm facing is reading custom plugin properties provided in the elasticsearch.yml file.

i would need to receive a list of properties, some of them are predefined, some of them are prefixed with "...my-custom.realm3.roles" but the postfix is an elasticsearch role name.

my-custom:
realm3:
order: 2
restResourceBaseUrl: "http://localhost:8080/restServices"
roles.all: "reporting_user"
roles.sl_elastic_admin: "watcher_admin,other_role"
roles.sl_elastic_viewer: "watcher_admin"

When elasticsearch is started with the plugin deployed it fails with a StartupException, please see full stacktrace at the bottom:

org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [xpack.security.authc.realms.my-custom.realm3.roles.sl_elastic_admin]

Code: public class CustomRealm extends Realm

Code: public class CustomSecurityExtensionPlugin extends Plugin implements ActionPlugin

Question:

• Is there a way to allow custom realm plugins to allow reading wildcard properties?
• What type of setting needs to be used for this?

Full Stacktrace:

[2019-09-23T10:38:25,028][INFO ][o.e.p.PluginsService ] [node-1] loaded plugin [elastic-security-wm6-custom-realm-plugin]
[2019-09-23T10:38:28,198][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [xpack.security.authc.realms.my-custom.realm3.roles.sl_elastic_admin] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.3.1.jar:7.3.1]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.3.1.jar:7.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.3.1.jar:7.3.1]
Caused by: java.lang.IllegalArgumentException: unknown setting [xpack.security.authc.realms.my-custom.realm3.roles.sl_elastic_admin] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:531) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:476) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:447) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:418) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:149) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.node.Node.(Node.java:357) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.node.Node.(Node.java:258) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:221) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.3.1.jar:7.3.1]
... 6 more

[TimV (Tim Vernum)]

You'll want to use a "group setting" wrapped in an "affix setting".

    private static final Setting.AffixSetting<Settings> ROLES_SETTING 
       = Setting.affixKeySetting(realmSettingPrefix(TYPE), "roles",
            (key) -> Setting.groupSetting(key + ".", Setting.Property.NodeScope));

Without the affix part, your group setting is registering a setting for xpack.security.authc.realms.my-custom.roles.* (without the realm name in it).
That would prevent you from having different settings per realm (which you might not need, but in general realms are supposed to support that).