Elasticsearch not recognizing valid syntax

[Evesy (Michael Eves)]

Hi,

I'm using the official elasticsearch cookbook to configure elasticsearch however the elasticsearch config yaml file generated when trying to configure the xpack.monitoring.exporters parameter is not being accepted by Elasticsearch and the service is failing to start:

Inside of the 'elasticsearch_configure' resource I have the below:

'xpack.monitoring.exporters' => "id1:
 type: http
 host: [\"#{node['elk_elasticsearch']['xpack']['monitoring']['export_url']}\"]",

This resource generates the following lines within elasticsearch.yml:

xpack.monitoring.exporters: |-
  id1:
   type: http
   host: ["localhost"]

Elasticsearch then fails to start with the below error:

java.lang.IllegalArgumentException: unknown setting [xpack.monitoring.exporters] did you mean [xpack.monitoring.enabled]?
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:278) ~[elasticsearch-5.1.1.jar:5.1.1]
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:246) ~[elasticsearch-5.1.1.jar:5.1.1]
    at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:138) ~[elasticsearch-5.1.1.jar:5.1.1]
    at org.elasticsearch.node.Node.<init>(Node.java:325) ~[elasticsearch-5.1.1.jar:5.1.1]
    at org.elasticsearch.node.Node.<init>(Node.java:229) ~[elasticsearch-5.1.1.jar:5.1.1]

If I remove the multiline characters, |-, then elasticsearch starts just fine. Both are valid yml so I'm not sure why it is failing to start.

This is using cookbook version 3.0.2, configuring elasticsearch 5.1

Any help would be appreciated.

Cheers

[colings86 (Colin Goodheart-Smithe)]

@Evesy

xpack.monitoring.exporters: |-
  id1:
   type: http
   host: ["localhost"]

You are right that this is valid YAML but it is not equivilent to:

xpack.monitoring.exporters:
  id1:
   type: http
   host: ["localhost"]

The first code block says 'set the value of the setting xpack.monitoring.exporters to the string:

'id1:
   type: http
   host: ["localhost"]'

The latter example says 'set xpack.monitoring.exporters.id1.type to http and xpack.monitoring.exporters.id1.host to `["localhost"].'

I think the reason it is doing this is that the value in

xpack.monitoring.exporters' => "id1:
 type: http
 host: [\"#{node['elk_elasticsearch']['xpack']['monitoring']['export_url']}\"]",

is surrounded in "'s so is being dumped as a string. I am not familar with the cookbook myself so I am not sure exactly what the fix is to allow the cookbook to interpret the value as structured settings rather than a string but you could either try removing the "'s:

'xpack.monitoring.exporters' => id1:
type: http
host: [\"#{node['elk_elasticsearch']['xpack']['monitoring']['export_url']}\"],

or maybe specify the settings separately:

could either try removing the "'s:

'xpack.monitoring.exporters.id1.type' => "http",
'xpack.monitoring.exporters.id1.host' => [\"#{node['elk_elasticsearch']['xpack']['monitoring']['export_url']}\"],

Hope that helps