Configuring puppet-elasticsearch with hiera yaml

Elastic Stack Elasticsearch
#1

Hello,

I've picked up a great little utility called wirbelsturm
(https://github.com/miguno/wirbelsturm). With it I've managed to automate
the creation of Vagrant backed VMs for a large chunk of my infrastructure
without much pain. Then I got to elasticsearch. I've tried a fe variations
of the below yaml, and cannot get elasticsearch to start.

I am running es on CentOS 6.5. What am I missing?

classes:

  • elasticsearch

elasticsearch::manage_repo: true
elasticsearch::repo_version: "1.3"
elasticsearch::java_install: true
elasticsearch::ensure: "present"
elasticsearch::status: "enabled"
elasticsearch::instance: { 'es-01': }

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dee408f2-6dda-4a81-914b-32be84c735e0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#2

I am also trying to setup elasticsearch cluster with puppet module using hiera.

Below is my hiera config:

classes:

  • elasticsearch
  • wget

elasticsearch::manage_repo: true
elasticsearch::repo_version: '2.x'

Elasticsearch Related Stuff

elasticsearch::version: '2.3.1-1'
elasticsearch::autoupgrade: false
elasticsearch::status: 'enabled'
elasticsearch::serviceprovider: 'init'
elasticsearch::datadir: '/data'

elasticsearch::config:
cluster:
name: 'Westeros'
bootstrap:
mlockall: true
index:
number_of_shards: '1'
number_of_replicas: '3'
refresh_interval: '5'
codec: 'best_compression'

Kibana Installation

kibana::version: '4.5.0'
kibana::plugins:

  • 'elasticsearch/marvel/2.3.1'
  • 'elasticsearch/shield/2.3.1'

So far I am manage to install elasticsearch successfully, but module seems like written in such a way that you can install multiple instances of elasticsearch on single node. But my requirement is to install on different hosts.

Still struggling to make it work.

#3

hello

I created a puppet configuration with hiera with the puppet module elasticsearch/elasticsearch from puppet forge on a centos 7 server

this is my profile

class profiles::elasticsearch {

class { 'elasticsearch':
ensure => present,
manage_repo => true,
repo_version => '2.x',
}

}

my hiera datas

elasticsearch::instances:
es-01:
config:
path.data: '/var/elasticsearch/'
node.name: 'bu'
cluster.name: 'namek'
discovery.zen.ping.unicast.hosts: '[10.0.0.2]'
index.store.type: 'mmapfs'
indices.memory.index_buffer_size: '30%'
index.translog.flush_threshold_ops: '50000'
indices.fielddata.cache.size: '40%'
network.host: '10.0.0.0'
init_defaults:
ES_HEAP_SIZE: '2g'

it creat a instance elasticsearch name es-01 the configuration is here
/etc/elasticsearch/es-01

i can manage my instance by the command
service elasticsearch-es-01 start|stop|status

the config file is
/etc/elasticsearch/es-01/elasticsearch.yml

MANAGED BY PUPPET

cluster:
name: namek
discovery:
zen:
ping:
unicast:
hosts: [10.0.0.2]
index:
store:
type: mmapfs
translog:
flush_threshold_ops: 50000
indices:
fielddata:
cache:
size: 40%
memory:
index_buffer_size: 30%
init_defaults:
ES_HEAP_SIZE: 2g
network:
host: 10.0.0.0
node:
name: bu
path:
data: /usr/share/elasticsearch/data/es-01
logs: /var/log/elasticsearch/es-01

I do not understand why my path.data is /usr/share/elasticsearch/data/es-01 and not /var/elasticsearch/es-01

#4

Hi @clement! Because the puppet module needs to also manage Elasticsearch data directories, the module assumes that you pass it the directory you want to store data in rather than defining path.data manually - if you give it the directory you want, it will handle creating the necessary yaml to give elasticsearch (and ensure the directory has right permissions, and so on).

You can do this either at the module level:

class { 'elasticsearch':
    datadir => '/var/elasticsearch',
}

Or per-instance in your hiera config:

elasticsearch::instances:
    es-01:
        datadir: /var/elasticsearch/es-01
#5

Hello tylerj

Many thanks it works very well now

#6

Hello,
thanks for sharing your config.

What module do you use for installing kibana? There is not official elastic repo for handing Kibana puppet installations. Am I wrong?

#7

Hi @Luca_Belluccini,

You can get the Kibana module from https://forge.puppet.com/jlambert121/kibana.

Hope that helps.

1 Like
#8

Thank you for the info!

Have a nice day

#9