Puppet-elasticsearch module not deploying elasticsearch.yml

Elastic Stack Elasticsearch
1

I am trying to manage the elasticsearch.yml file at the default location of /etc/elasticsearch/elasticsearch.yml

The following works fine, but isn't what I want:

class { 'elasticsearch':
    version => '6.2.4',
    status => 'disabled',
    datadir_instance_directories => false,
    instances => {
      $facts['hostname'] => {
        'config' => {
          'network.host' => '_site_',
          'node.name'    => $facts['hostname'],
        }
      }
    }
  }

It creates a file at /etc/elasticsearch/elastic-node01/elasticsearch.yaml

I then tried the following configuration (which from the documentation I read, I think should work, but it isn't working)

class { 'elasticsearch':
    version => '6.2.4',
    status => 'disabled',
    datadir_instance_directories => false,
    config => {
      'cluster' => {
        'network.host'     => '_site_',
        'node.name'        => $facts['hostname'],
      }
    }
  }

Both configurations run without errors, but in the second configuration, no file is created at /etc/elasticsearch/elasticsearch.yml, in fact, puppet actually spits out the following:

Notice: /Stage[main]/Elasticsearch::Package/Package[elasticsearch]/ensure: created
Info: /Stage[main]/Elasticsearch::Package/Package[elasticsearch]: Scheduling refresh of Exec[remove_plugin_dir]
Notice: /Stage[main]/Elasticsearch::Package/Exec[remove_plugin_dir]: Triggered 'refresh' from 1 event
Notice: /Stage[main]/Elasticsearch::Config/File[/usr/share/elasticsearch/plugins]/ensure: created
Notice: /Stage[main]/Elasticsearch::Config/File[/etc/elasticsearch/scripts]/ensure: created
Info: Computing checksum on file /etc/elasticsearch/elasticsearch.yml
Info: /Stage[main]/Elasticsearch::Config/File[/etc/elasticsearch/elasticsearch.yml]: Filebucketed /etc/elasticsearch/elasticsearch.yml to puppet with sum 3aa77775c113d3828a9e8e87637eb286
Notice: /Stage[main]/Elasticsearch::Config/File[/etc/elasticsearch/elasticsearch.yml]/ensure: removed
Info: Computing checksum on file /etc/elasticsearch/jvm.options
Info: /Stage[main]/Elasticsearch::Config/File[/etc/elasticsearch/jvm.options]: Filebucketed /etc/elasticsearch/jvm.options to puppet with sum 775a7b047687f7389149bff042a0e04d
Notice: /Stage[main]/Elasticsearch::Config/File[/etc/elasticsearch/jvm.options]/ensure: removed
Info: Computing checksum on file /etc/elasticsearch/log4j2.properties
Info: /Stage[main]/Elasticsearch::Config/File[/etc/elasticsearch/log4j2.properties]: Filebucketed /etc/elasticsearch/log4j2.properties to puppet with sum b3030447ca9e02e0e19b2c3d38bea137
Notice: /Stage[main]/Elasticsearch::Config/File[/etc/elasticsearch/log4j2.properties]/ensure: removed

Any suggestions?

I am using module version 6.3.3 with Puppet 5.3

2

Hi @Alex_L,

Check /etc/elasticsearch/$facts['hostname']/. The puppet module is done so that you can run more than one Elasticsearch instance per machine.

An example from one of my machines that have two instances ( es-00 and es-01)

# find /etc/elasticsearch/
...
/etc/elasticsearch/es-00
/etc/elasticsearch/es-00/scripts
/etc/elasticsearch/es-00/logging.yml
/etc/elasticsearch/es-00/jvm.options
/etc/elasticsearch/es-00/log4j2.properties
/etc/elasticsearch/es-00/elasticsearch.yml
/etc/elasticsearch/es-00/ingest-geoip
/etc/elasticsearch/es-00/ingest-geoip/GeoLite2-Country.mmdb
/etc/elasticsearch/es-00/ingest-geoip/GeoLite2-ASN.mmdb
/etc/elasticsearch/es-00/ingest-geoip/GeoLite2-City.mmdb
/etc/elasticsearch/es-00/elasticsearch.keystore
/etc/elasticsearch/es-01
/etc/elasticsearch/es-01/scripts
/etc/elasticsearch/es-01/logging.yml
/etc/elasticsearch/es-01/jvm.options
/etc/elasticsearch/es-01/log4j2.properties
/etc/elasticsearch/es-01/elasticsearch.yml
/etc/elasticsearch/es-01/ingest-geoip
/etc/elasticsearch/es-01/ingest-geoip/GeoLite2-Country.mmdb
/etc/elasticsearch/es-01/ingest-geoip/GeoLite2-ASN.mmdb
/etc/elasticsearch/es-01/ingest-geoip/GeoLite2-City.mmdb
/etc/elasticsearch/es-01/elasticsearch.keystore
...

It comes from https://github.com/elastic/puppet-elasticsearch/blob/master/manifests/instance.pp#L136

3

Thanks for your answer. Based on what you have said, I don't think I explained myself properly. I only want to run one instance of elasticsearch on the server. I want to use the puppet module to install based on the default elastic configuration i.e: /etc/elasticsearch/elasticsearch.yml and not /etc/elasticsearch/my-instance-name/elasticsearch.yml

4

Then this is not the puppet module you are looking for :wave:

I use the same module for machines where I run just one Elasticsearch instance. The config dir I use is /etc/elasticsearch/es-00/

Maybe there are properties to the puppet module I don't know of...

The README.md does state

This module works with the concept of instances. For service to start you need to specify at least one instance.

5

Thank you for pointing that out to me.
I realize now that I can still use this module for a single instance. Just need to adapt to the new locations of all the files and system services etc.

6

The module creates individual SystemD unit files for each instance. I have only needed to customize a few things to get ES ingest nodes with specific plugins to work (especially on machines that run several ES nodes but over all I am very happy with the puppet module :slight_smile:

7

Howdy! The confusion and non-standard setup here is one reason why we're probably going to drop support for multiple instances, since it's the source of lots of bugs. If there's specific feedback about whether this does/does not seem like a good idea to keep or drop, we'd love feedback on the aforementioned github issue. :+1:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.