Java.lang.IllegalArgumentException with environment variable placeholder

Elastic Stack Elasticsearch
1

Context of environment: Newbee engaged with the learning and building of a current-version single node elasticsearch-kibana-logstash stack with manually configured security, within a current virtual Ubuntu machine.

Context of Problem: Running and re-running bin/elasticsearch-certutil and -keystore tools, regardless of if installed elasticsearch is running or not.

The yet-unresolved issue: At some point after running a bin/elasticsearch- tool, Java throws an IllegalArgumentException: "Could not resolve placeholder 'ES_PATH_CONF' which prevents any further successful engagement with those elasticsearch- tools.

Now I do know what ES_PATH_CONF is, what it should be, and where it is defined. But I have no clue why Java cares.

What have I done to get out of this so far? Go back to a prior-saved virtual machine state where the problem has not yet shown its face.

Attempts to prevent this issue:

  1. run the bin/elasticsearch- tool with "sudo -E ..." rather than with just "sudo ...". This seemed to enable me to run and rerun the tools for a longer period of time before the Exception starts.

Attempts to clear this issue:

  1. Rebooting the system, which does not clear the issue, which continues to stop further use of the -certutil and -keystore elasticsearch tools.

Full Exception from "sudo -E ./bin/elasticsearch-keystore list":
'Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'ES_PATH_CONF''
'at org.elasticsearch.common.settings.PropertyPlaceholder.parseStringValue(PropertyPlaceholder.java:102)'
'at org.elasticsearch.common.settings.PropertyPlaceholder.replacePlaceholders(PropertyPlaceholder.java:57)'
'at org.elasticsearch.common.settings.Settings$Builder.replacePropertyPlaceholders(Settings.java:1314)'
'at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:56)'
'at org.elasticsearch.common.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:110)'
'at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:54)'
'at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:85)'
'at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:94)'
'at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:85)'
'at org.elasticsearch.cli.Command.main(Command.java:50)'
'at org.elasticsearch.launcher.CliToolLauncher.main(CliToolLauncher.java:64)'

Full Exception from "sudo -E ./bin/elasticsearch-keystore http":
'Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'ES_PATH_CONF''
'at org.elasticsearch.common.settings.PropertyPlaceholder.parseStringValue(PropertyPlaceholder.java:102)'
'at org.elasticsearch.common.settings.PropertyPlaceholder.replacePlaceholders(PropertyPlaceholder.java:57)'
'at org.elasticsearch.common.settings.Settings$Builder.replacePropertyPlaceholders(Settings.java:1314)'
'at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:56)'
'at org.elasticsearch.common.cli.EnvironmentAwareCommand.createEnv(EnvironmentAwareCommand.java:110)'
'at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:54)'
'at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:85)'
'at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:94)'
'at org.elasticsearch.xpack.security.cli.CertificateTool.execute(CertificateTool.java:161)'
'at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:85)'
'at org.elasticsearch.cli.Command.main(Command.java:50)'
'at org.elasticsearch.launcher.CliToolLauncher.main(CliToolLauncher.java:64)'

Any suggestions in what direction I should start looking to learn what phenomena is setting me up for this IllegalArgumentException would be much appreciated!
Thank you for your time and consideration!

2

Regardless of knowing why, this phenomena can be created with edits to the /etc/elasticsearch/elasticsearch.yml file by inserting a prepend of "${ES_PATH_CONF}/" to entries containing "certs/".

Neither full path prepends of "/etc/elasticsearch/" nor the untouched installed settings with "certs/" create the phenomena when going to execute tools: elasticsearch-certutil, elasticsearch-keystore, regardless of the installed elasticsearch running or not.

Go ahead, have some fun and give it a try!

3

This error means that the ES_PATH_CONF variable is not set. To fix it, set the environment variable. I opened #110858 because I think we could make this error message clearer.

4

Thank you for looking in to this.

After adding '${ES_PATH_CONF}/' as a prepend to 'certs/....' in the elasticsearch.yml ssl configurations, you will get the same java IllegalArgumentException regardless of the ES_PATH_CONF environment variable being set or not.

These edits to elasticsearch.yml do not require restarting the elasticsearch service to see the IllegalArgumentException when going to execute elasticsearch-certutil or elasticsearch-keystore.

You can toggle back and forth to success or IllegalArgumentException failure by simply adding or removing the '${ES_PATH_CONF}/' prepend to 'certs/...' elasticsearch.yml configurations.

5

I can't reproduce this. When I set ${ES_PATH_CONF} I do not get this error.