Error checking of configuration files at startup?

Hi,

Since we've been discussing on the Solr list how to handle
configuration errors at startup (see
http://markmail.org/thread/ywkfmxjboyixkrjc), I was also curious how
ElasticSearch handles this.

So I installed 0.16, and intentionally put two typos in my
config/elasticsearch.yml:

Cluster Settings

cluster:
nameX: elasticsearch
foobar

The foobar indeed causes an error message and elasticsearch refuses to
start (a good thing, I think):

  • SettingsException[Failed to load settings from
    [file:/home/mike/src/elasticsearch-0.16.0/config/elasticsearch.yml]]
    ScannerException[while scanning a simple key; could not found expected ':']

Though, that message doesn't tell me the line number...

However, if I remove the foobar and leave the "nameX: ..." line,
there's no erorr nor warning. Is that expected? I think it would be
better if mis-spellings were treated as a hard error (server doesn't
start) so users don't incorrectly think they've configured something?

Mike

http://blog.mikemccandless.com

Agreed, it would be great to handle such mis configuration (i.e. nameX and not name), though it gets tricky when it comes to building a modular system with custom possible plugins (i.e. what part of the system each module is "responsible" to validate, and what are the valid setting names are).

Its not done in elasticsearch, but, corresponding to each module config, one can set it to DEBUG level in the logging, and see exactly the configuration values set for it (and check if the settings took place). Not optimal, but thats what we have today.
On Tuesday, May 17, 2011 at 3:38 PM, Michael McCandless wrote:

Hi,

Since we've been discussing on the Solr list how to handle
configuration errors at startup (see
http://markmail.org/thread/ywkfmxjboyixkrjc), I was also curious how
Elasticsearch handles this.

So I installed 0.16, and intentionally put two typos in my
config/elasticsearch.yml:

Cluster Settings

cluster:
nameX: elasticsearch
foobar

The foobar indeed causes an error message and elasticsearch refuses to
start (a good thing, I think):

  • SettingsException[Failed to load settings from
    [file:/home/mike/src/elasticsearch-0.16.0/config/elasticsearch.yml]]
    ScannerException[while scanning a simple key; could not found expected ':']

Though, that message doesn't tell me the line number...

However, if I remove the foobar and leave the "nameX: ..." line,
there's no erorr nor warning. Is that expected? I think it would be
better if mis-spellings were treated as a hard error (server doesn't
start) so users don't incorrectly think they've configured something?

Mike

http://blog.mikemccandless.com

Thanks Shay.

We have the same challenge in Solr, where because plugins can "claim"
arbitrary configuration, misconfiguration can easily remain
undetected...

I think somehow a plugin should explicitly "claim" which settings it
has validated. Then at the end of initializing all plugins, if any
configured settings were not claimed by any plugins, that's a hard
error. This would require more work on plugin-writers' part, but it
means users of those plugins get a tighter experience, knowing if
they've misconfigured something.

Mike

http://blog.mikemccandless.com

On Tue, May 17, 2011 at 2:04 PM, Shay Banon
shay.banon@elasticsearch.com wrote:

Agreed, it would be great to handle such mis configuration (i.e. nameX and
not name), though it gets tricky when it comes to building a modular system
with custom possible plugins (i.e. what part of the system each module is
"responsible" to validate, and what are the valid setting names are).
Its not done in elasticsearch, but, corresponding to each module config, one
can set it to DEBUG level in the logging, and see exactly the configuration
values set for it (and check if the settings took place). Not optimal, but
thats what we have today.

On Tuesday, May 17, 2011 at 3:38 PM, Michael McCandless wrote:

Hi,

Since we've been discussing on the Solr list how to handle
configuration errors at startup (see
http://markmail.org/thread/ywkfmxjboyixkrjc), I was also curious how
Elasticsearch handles this.

So I installed 0.16, and intentionally put two typos in my
config/elasticsearch.yml:

Cluster Settings

cluster:
nameX: elasticsearch
foobar

The foobar indeed causes an error message and elasticsearch refuses to
start (a good thing, I think):

  • SettingsException[Failed to load settings from
    [file:/home/mike/src/elasticsearch-0.16.0/config/elasticsearch.yml]]
    ScannerException[while scanning a simple key; could not found expected ':']

Though, that message doesn't tell me the line number...

However, if I remove the foobar and leave the "nameX: ..." line,
there's no erorr nor warning. Is that expected? I think it would be
better if mis-spellings were treated as a hard error (server doesn't
start) so users don't incorrectly think they've configured something?

Mike

http://blog.mikemccandless.com

Yea, thats the logic I was thinking about as well.
On Tuesday, May 17, 2011 at 10:30 PM, Michael McCandless wrote:

Thanks Shay.

We have the same challenge in Solr, where because plugins can "claim"
arbitrary configuration, misconfiguration can easily remain
undetected...

I think somehow a plugin should explicitly "claim" which settings it
has validated. Then at the end of initializing all plugins, if any
configured settings were not claimed by any plugins, that's a hard
error. This would require more work on plugin-writers' part, but it
means users of those plugins get a tighter experience, knowing if
they've misconfigured something.

Mike

http://blog.mikemccandless.com

On Tue, May 17, 2011 at 2:04 PM, Shay Banon
shay.banon@elasticsearch.com wrote:

Agreed, it would be great to handle such mis configuration (i.e. nameX and
not name), though it gets tricky when it comes to building a modular system
with custom possible plugins (i.e. what part of the system each module is
"responsible" to validate, and what are the valid setting names are).
Its not done in elasticsearch, but, corresponding to each module config, one
can set it to DEBUG level in the logging, and see exactly the configuration
values set for it (and check if the settings took place). Not optimal, but
thats what we have today.

On Tuesday, May 17, 2011 at 3:38 PM, Michael McCandless wrote:

Hi,

Since we've been discussing on the Solr list how to handle
configuration errors at startup (see
http://markmail.org/thread/ywkfmxjboyixkrjc), I was also curious how
Elasticsearch handles this.

So I installed 0.16, and intentionally put two typos in my
config/elasticsearch.yml:

Cluster Settings

cluster:
nameX: elasticsearch
foobar

The foobar indeed causes an error message and elasticsearch refuses to
start (a good thing, I think):

  • SettingsException[Failed to load settings from
    [file:/home/mike/src/elasticsearch-0.16.0/config/elasticsearch.yml]]
    ScannerException[while scanning a simple key; could not found expected ':']

Though, that message doesn't tell me the line number...

However, if I remove the foobar and leave the "nameX: ..." line,
there's no erorr nor warning. Is that expected? I think it would be
better if mis-spellings were treated as a hard error (server doesn't
start) so users don't incorrectly think they've configured something?

Mike

http://blog.mikemccandless.com