ECE AWS automation help

[tarp (Tim Arp)]

Hi,

With the latest 1.01 version of ECE, the installation tokens have been introduced. I have been reading, https://www.elastic.co/guide/en/cloud-enterprise/current/ece-installation-script.html
But I think there are some features missing from the install script.

I'm building this at AWS with 4 CloudFormation stacks.
1 Coordinator-Init (this just is the first node, to be removed later)
AutoScale Coordinators
AutoScale Allocators
AutoScale Proxies

I had intended that each of the 3 Autoscale groups would just build and pass a DNS name as the coordinator host and build into the system. All totally automated. Now I have the tokens in 1.01 to deal with.
I can pass the tokens around in encrypted S3 buckets between the stacks. That is doable.
On the first install of the Coordinator-Init server, would there be an easier way to pull out the tokens rather than parsing the script output? I have a feeling the script output could change. Maybe a flag to write the tokens to a file that I could then copy to S3 for the other builds.
Any ideas how to improve this so it can be automated other than my idea here?

Thanks,
Tim

[nrichers (Nik Richers)]

Hi @tarp,

One option you might consider is to generate additional tokens for the roles you need through the ECE RESTful API and parsing the JSON output to get the token itself. That approach should be much more reliable than parsing the output of the installation script, which may well change.

For automation purposes, you will need to generate at least some tokens through the RESTful API no matter what, as I don't think the installation process on the first host outputs a token that will let you create a proxy without also going into the Cloud UI (you get a basic token with no role permissions, an allocator token, and an emergency coordinator token).

For example:

curl -H 'Content-Type: application/json' -u root:password http://localhost:12443/api/v1/platform/configuration/security/enrollment-tokens -d '{ "persistent": true, "roles": [ "allocator"] }'
{
"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI3OTk4MDNmYi0wNjg1LTQ5MzYtYjEwMy00Njk5MWZiMWQ5YjEiLCJyb2xlcyI6WyJhbGxvY2F0b3IiXSwiaXNzIjoiY3VycmVudCIsInBlcnNpc3RlbnQiOnRydWV9.dpk4zaR6ePcaiojmOtSe8DCezK5wKnbRgwfxHG5q14M",
"token_id": "799803fb-0685-4936-b103-46991fb1d9b1"
}

See Generate roles tokens | Elastic Cloud Enterprise Reference [3.6] | Elastic for more details. There's also a recent blog post on our API that includes a section on these tokens that I borrowed the example from: Exploring the API for Elastic Cloud Enterprise | Elastic Blog (last section, called "Dipping a toe into automation ...").

I hope this helps. Let me know if you have any additional questions or if I did not address your post adequately.

Nik

[tarp (Tim Arp)]

Hi Nik,

Thanks for the reply. After I wrote that I did find myself learning more about the API to generate tokens. You are right, that probably is the way to go. I will still need to scrape the root initial password off the install output. The way we are designing our cloud resources is to never have interaction with the EC2 instance from the command line. So I may still look at encrypting the password and placing it on S3 via a script, then to be picked up by another script that will generate tokens and install the other components. A lot to think about.

Thanks,
Tim

[Alex_Piggott]

@tarp

All of the secrets (including the tokens, eg allocator_only_token) are written into /mnt/data/elastic/bootstrap-state/bootstrap-secrets.json so you can use eg jq to parse them out

(edit: see https://www.elastic.co/guide/en/cloud-enterprise/current/ece-installing.html#ece-installing-first)

Alex

[nrichers (Nik Richers)]

@Alex_Piggott ,

Our docs used to include the info about retrieving tokens via jq but we removed that info from the docs, as we wanted to limit the number of cases where we promote the use of non-standard tools - the API can be used to create new tokens if you need them.

We do still document how to retrieve some of the passwords via jq, including the root password Tim needs, see https://www.elastic.co/guide/en/cloud-enterprise/current/ece-retrieve-passwords.html.

Nik