My undetstanding is, any missing entries (like passwords) in the yml will be retrieved from the keystore.
Having an entry in the keystore should not cause the cluster failure.
Eg: Having a "xpack.security.authc.realms.ldap1.secure_bind_password" in the keystore is causing the elasticsearch cluster failure with the error:
" Suppressed: java.lang.IllegalArgumentException: missing realm type [xpack.security.authc.realms.ldap1.type] for realm"
in your keystore, you'd have an LDAP realm in your elasticsearch.yml that is named ldap1, because otherwise you wouldn't need to set xpack.security.authc.realms.ldap1.secure_bind_password.
Is there any specific use case for you to set a property for a realm that doesn't exist ?
If we look at this from a large scale deployment standpoint, it is possible that people create various keys in the keystore and expect that the process picks only those necessary for it or those specified in the YAML.
When you are designing a keystore, it should only be constrained to its responsibility of storing keys. Tying the responsibility of being an additional source of truth along with the responsibility of being a keystore, does not sound proper to me (considering the SOLID principles).
There is some ongoing discussion about how we could change the behavior of the keystore with regards to invalid settings that you can keep track of here
We were migrating to new LDAP and added the new settings related to that in the yml. We removed the old LDAP settings from the yml. But, I did not remove the old LDAP password entry from the keystore. I did not expect or anticipate the cluster failure due to an entry in keystore.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.