Elasticsearch puppet host config

Elastic Stack Elasticsearch
1

Hello Everyone,

I'm working on getting elasticsearch to work using puppet. I've been able to get it installed but not to get it to accept connections.

This is my current puppet elasticsearch manifest:

class essearch{
        yumrepo {'elasticsearch':,
                descr => "Local Elasticsearch yum repo",
                baseurl => "ftp://localrepo/pub/repo/elasticsearch",
                enabled => 1,
                gpgcheck => 0,
        }->
        class { 'elasticsearch':
                java_install            => true,
                manage_repo             => false,
                restart_on_change       => true,
                repo_version            => '5.x',
                jvm_options             => [
                        '-Xms24g',
                        '-Xmx24g',
                ],
        }
        elasticsearch::instance { 'searchbpe':
                        datadir => '/data/elDATA',
                        config => {
                                        'network' => {
                                                        'host' => [$::ipaddress, '127.0.0.1' ],
                                        },
                                        'cluster' => {
                                                        'name' => 'searchbpe',
                                        },
                                        'index' => {
                                                        'number_of_replicas' => 1,
                                        },
                                        'discovery' => {
                                                        'zen' => {
                                                                        'ping' => {
                                                                                        'unicast' => {
                                                                                                        'hosts' => ["host1","host2"]
                                                                                        },
                                                                        },
                                                        },
                                        },
                                        'node' => {
                                                'name' => $hostname
                                        },
                        }
        }
        firewall { '400 allow elasticsearch access for any':
                        port   => [9200, 9300],
                        proto  => tcp,
                        action => accept,
        }
        file { ["/data"]:
                        ensure => "directory",
                        owner => 'root',
                        group => 'root',
        }
}

When I start elasticsearch on host1 and check the ports listing I get this:

tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      23567/java
tcp6       0      0 ::1:9200                :::*                    LISTEN      23567/java
tcp6       0      0 127.0.0.1:9300          :::*                    LISTEN      23567/java
tcp6       0      0 ::1:9300                :::*                    LISTEN      23567/java

Does anyone have an idea what i'm doing wrong here? Because if I make the config file /etc/elasticsearch/elasticsearch.yml with this content it works:

#### Managed by Puppet ####
cluster.name: searchbpe
discovery.zen.ping.unicast.hosts:
  - host1
  - host2
network.host:
  - 123.456.789.0
  - "127.0.0.1"
node.name: host2
path.data: /data/elDATA
path.logs: /var/log/elasticsearch/searchbpe

Thanks for any feedback.

2

Hi @meaglin,

The puppet module manages individual instances with their configurations stored in a location (such as in your case) at /etc/elasticsearch/searchbpe/elasticsearch.yml. What does that file contain for you? The module should start the necessary daemon with service/systemctl as appropriate for whatever distro you're on (for example, in the case of CentOS 7, the module creates an elasticsearch-searchbpe service.

3

Hi Tylerjl,

I'm running Oracle Linux 7, so it should be the same as CentOS 7.

The config file contains:
### MANAGED BY PUPPET ###
---
cluster.name: searchbpe
discovery.zen.ping.unicast.hosts:
- host1
- host2
index.number_of_replicas: 1
network.host:
- "123.456.789.0"
- "127.0.0.1"
node.name: host1
path.data: /data/elDATA
path.logs: /var/log/elasticsearch/searchbpe

I was wondering what you meant with the last thing about the service, when looking into it a bit i found a service with this name elasticsearch-searchbpe.service, but it won't start, now i have something to check.

Thanks for the response, maybe i will have further questions.

4

Yeah, the module operates under the notion of instances so Puppet manages individual services on a machine, not the generic elasticsearch service (this is so that multiple instances can be easily managed on very large machines if someone wants to run more than just one Elasticsearch service).

Note that in the case of Oracle Linux 7/CentOS 7, often times daemon startup stdout/stderr may not show up in systemctl status elasticsearch-searchpbe or in Elasticsearch log files. You can more easily find Elasticsearch startup stdout/stderr output in this case by searching through the journal (journalctl).

5

Hi Tylerjl,

I'm having an issue with the kibana module now and i'm not sure if it's me but i think it's not.

I get this message from puppet:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Syntax error at 'variant'; expected ')' at /etc/puppet/environments/production/modules/kibana/manifests/init.pp:31

I've checked it and with is the part of the file where it goes wrong, but i'm not seeing any problem. I'm thinking of installing Kibana via a yum command from puppet and put in a config file.
The error points to the first line of this part:

class kibana (
  variant[Enum['present', 'absent', 'latest'], Pattern[/^\d([.]\d+)*(-[\d\w]+)?$/]] $ensure = 'present',
  hash[String[1], Variant[String[1], Integer, Boolean, Array]]           $config            = {},
  boolean                                                                $manage_repo       = true,
  string                                                                 $repo_key_id       = '46095ACC8548582C1A2699A9D27D666CD88E42B4',
  string                                                                 $repo_key_source   = 'https://artifacts.elastic.co/GPG-KEY-elasticsearch',
  optional[Integer]                                                      $repo_priority     = undef,
  optional[String]                                                       $repo_proxy        = undef,
  variant[Enum['5.x'], Pattern[/^4\.(1|[4-6])$/]]                        $repo_version      = '5.x',
)
6

What version of Puppet are you on? The puppet-kibana module is only compatible with 4.x versions of Puppet.

7

That explains, i'm still on the 3.x version. Then i will go with the other option.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.