Puppet - Found multiple default providers for elasticsearch_license

[bilsch (Bill)]

We are in the process of upgrading our puppet module from ~3 years ago with some manual shield hacks to the elastic supported module.

Using puppet 4.5.3 ( pe ) , module is 6.3 , host os is rhel 7

I'm getting an error - Found multiple default providers for elasticsearch_license

Not sure how/why. I have set the security_plugin at the elasticsearch class level as well as within the hiera instance.

I tried adding the x-pack to plugins but got an error as this plugin is packaged with elasticsearch ( thank you for that )

Any thoughts? I'm either missing something obvious or there is a module bug

[tylerjl (Tyler Langlois)]

Hey there, thank you for using the updated module and using the forums!

The elasticsearch_license type and provider have their provider set from the elasticsearch::security_plugin which you indicate you've set correctly. Typically that message you're highlighting isn't an error, however - even if multiple providers are available for the type, Puppet will choose one (and should choose the one that is specified in security_plugin). Do you perhaps have the complete run output to determine if there's a different error messages returned from Puppet?

[bilsch (Bill)]

Tyler,

I had to change a few things to prevent leaking info. Assuming you are able to access the support issue I can upload details there ( 00235706 ). The output is too large for posting here, pastebin It expires in a week.

I have debug and other output as well. This is running vagrant provision with a puppet provisioner and no master.

It appears related to roles and users. In the debug output it looks like it wants to append to the roles and users files in /etc/elasticsearch/x-pack - should that go to the instance directory instead?

host yaml:

elasticsearch::users:
  logstash:
    password: "%{hiera('defaults::elasticsearch::logstash_pass')}"
    # roles: ['logstash']
  "%{hiera('profile::elasticsearch::admin_user')}":
    password: "%{hiera('profile::elasticsearch::admin_password')}"
elasticsearch::roles:
 admin:
   privileges:
     cluster: all
     index:
       '*': all

[tylerjl (Tyler Langlois)]

Thanks @bilsch - I can hop into the support case if you'd like to drop anything sensitive here, otherwise we can carry on debugging here, whichever you prefer.

The catalog output seems to point at blaming elasticsearch_user_roles resources which appear to be failing and blocking the rest of the Elasticsearch module from proceeding. This native type just alters the users_roles file to add users to the necessary roles, which exists at /etc/elasticsearch/x-pack/users_roles in the pre-6.3 version of x-pack, or /etc/elasticsearch/users_roles in versions 6.3 onward. In your setup, are you running Elasticsearch 6.3 or pre-6.3, and do you have any pre-existing X-Pack files in these locations? If you need to post the manifest in your support case, that works as well.

[bilsch (Bill)]

This is elastic 6.3.

I nuked the previous VM and oddly enough its sorta happy now. The users and roles create correctly but I'm having issues with permissions and/or ssl ( guessing the role content needs adjusting for 6, I'm upgrading from 1.7 shield )

==> elp: Error: /Stage[main]/Elasticsearch::License/Elasticsearch_license[xpack]: Could not evaluate: Elasticsearch API responded with: action [cluster:admin/xpack/license/put] is unauthorized for user [admin]

My suspicion is that I nuked /etc/elasticsearch/elk dir and expected puppet to put that stuff back. Going to continue down the happy path for a bit to make progress on something. I will try to re-create the scenario later.

[tylerjl (Tyler Langlois)]

Gotcha - for what it's worth, the native types will try and figure out which file they need to modify, but if you happen to have shield and x-pack security files both present, the types may get confused - if that does happen, you can always force the type to talk to the specific files for the type of security plugin you're using:

Elasticsearch_user_roles {
  provider => 'oss_xpack'
}

(For >= 6.3, at least - <= 6.2 provider would be xpack, etc.)

[bilsch (Bill)]

Awesome. I appear to be working again Sorry for the confusion.

So, any thoughts if that puppet module can support 1.7, 2.[2,4] and 6.3 together? I know supporting such old ancient versions is a pita so don't feel obligated to say yes but ... it would be amazing!

[tylerjl (Tyler Langlois)]

The only guaranteed (read: tested) configurations that I can offer are the still-supported versions noted in the product end-of-life matrix, so while currently supported versions should certainly work with the module, anything outside that scope is subject to breakage at any time.

With that being said, if you look at the Travis tests, you'll notice that I haven't gotten around to removing 2.x from my tests and those still appear to pass, so it looks like 2.x still works, albeit as a coincidence and not explicitly as a supported effort. I really don't have any insight into 1.x, though I would have to imagine that some of the module changes are probably incompatible, though you're welcome to try! Off-hand I know that the module relies on things that aren't present in some 1.x installations like plugin-descriptor.properties files for plugins so I would think plugins on 1.x probably don't work well on recent versions of the module.