Keystore for secrets in elastic-agent.yml

Elastic Observability
1

I have some secret values in my ./elastic-agent.yml file. I was hoping to use a keystore to help manage those secrets.

I tried this command ./elastic-agent keystore add outputs.default.password, but it gave the error

Error: unknown command "keystore" for "elastic-agent"

What did I do wrong? What's the correct way to set up the elastic agent keystore?

2

Hi,

the Elastic Agent does not support the use of a keystore for managing secrets like the Elasticsearch does.

However, you can use environment variables to manage your secrets.
Here's an example of how you can do this:

Set your secret as an environment variable. For example, in a Unix-based system, you can use the export command:

export OUTPUTS_DEFAULT_PASSWORD=yourpassword

In your elastic-agent.yml file, reference the environment variable using the ${} syntax:

outputs:
  default:
    password: '${OUTPUTS_DEFAULT_PASSWORD}'

When the Elastic Agent reads the configuration file, it will replace ${OUTPUTS_DEFAULT_PASSWORD} with the value of the OUTPUTS_DEFAULT_PASSWORD environment variable.

Remember to restart the Elastic Agent after setting the environment variables for the changes to take effect.

Regards

4

Thank you for your explanation.

What I liked about keystores for kibana, logstash, beats etc... was that it encodes the secrets so that they are not human readable.

I guess that won't be possible with elastic-agents then? Unless I find a way to encode the .env file through other 3rd party tools and decode them when the elastic agent starts up?

5

It is possible if you use the Fleet Managed agents, but not if using Standalone agents.

When you use Fleet manage agents, the configuration deployed on the agents is encrypted and is managed from the Fleet Server.

6

Thank you. I'm actually trying to follow along this blog which uses docker-compose to set up elastic-agent, apm and fleet-server.

And in the docker-compose.yml file,

There is specifically these lines that set up the container for elastic-agent, fleet-server and apm:

  fleet-server:
    depends_on:
      kibana:
        condition: service_healthy
      es01:
        condition: service_healthy
    image: docker.elastic.co/beats/elastic-agent:${STACK_VERSION}
    volumes:
      - certs:/certs
      - fleetserverdata:/usr/share/elastic-agent
      - "/var/lib/docker/containers:/var/lib/docker/containers:ro"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "/sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro"
      - "/proc:/hostfs/proc:ro"
      - "/:/hostfs:ro"
    ports:
      - ${FLEET_PORT}:8220
      - ${APMSERVER_PORT}:8200
    user: root
    environment:
      - SSL_CERTIFICATE_AUTHORITIES=/certs/ca/ca.crt
      - CERTIFICATE_AUTHORITIES=/certs/ca/ca.crt
      - FLEET_CA=/certs/ca/ca.crt
      - FLEET_ENROLL=1
      - FLEET_INSECURE=true
      - FLEET_SERVER_ELASTICSEARCH_CA=/certs/ca/ca.crt
      - FLEET_SERVER_ELASTICSEARCH_HOST=https://es01:9200
      - FLEET_SERVER_ELASTICSEARCH_INSECURE=true
      - FLEET_SERVER_ENABLE=1
      - FLEET_SERVER_CERT=/certs/fleet-server/fleet-server.crt
      - FLEET_SERVER_CERT_KEY=/certs/fleet-server/fleet-server.key
      - FLEET_SERVER_INSECURE_HTTP=true
      - FLEET_SERVER_POLICY_ID=fleet-server-policy
      - FLEET_URL=https://fleet-server:8220
      - KIBANA_FLEET_CA=/certs/ca/ca.crt
      - KIBANA_FLEET_SETUP=1
      - KIBANA_FLEET_USERNAME=elastic
      - KIBANA_FLEET_PASSWORD=${ELASTIC_PASSWORD}
      - KIBANA_HOST=https://kibana:5601

What I wanted to do was actually have this container also depends_on: elastic_agent_setup whereby elastic_agent_setup is a new container that runs a bash command to the effect of echo 'superuser-password' | ./elastic-agent keystore add outputs.default.password --stdin. Then I copy that the keystore to a volume that can be accessed by my fleet-server container such that my fleet-server container can avoid the environment variable KIBANA_FLEET_PASSWORD=${ELASTIC_PASSWORD}.

Am I correct in saying that the fleet-server container is a fleet managed agent, and there is some way idiomatic way to encrypt the ELASTIC_PASSWORD that's used to set the KIBANA_FLEET_PASSWORD when using docker-compose?

Thanks!

7

Which part of the tutorial mentions editing the elastic-agent.yml file? Couldn't find it.

It uses Fleet Server, which means that the agents are managed by Fleet, you do not have to edit any agent configuration file.

8

I see. I think the main issue is that $ELASTIC_PASSWORD mentioned in the containers environment variable KIBANA_FLEET_PASSWORD=${ELASTIC_PASSWORD} is stored in the .env file in the same directory as the docker-compose.yml file. And the .env has all values that are not encrypted , they are in plain human read-able text. What's the best way to make secrets used by fleet-server container not human readable?

9

It depends entirely on your infrastructure, on how you are going to deploy your stack, this is unrelated to Elastic.

For example if you use Docker Swarm (not just compose) you can create and use secrets.

But this is out of the scope of the forum.

10

That's correct. Because I wasn't using swarm, i assumed i was only left with 2 options:

a) create a setup container that will run the ./elastic-agent keystore add ... prior to starting the fleet-server container, as described in the main question at the top

or

b) use a completely different 3rd party tool to encrypt the .env file , which would have nothing to do with elasticsearch

The original question in this post was to see if a) is a possibility, or if there are other idiomatic ways in elasticsearch to handle the situation.

And i believe everyone is saying a) is not possible.

11

Elastic Agent does not have any Keystore support, so the command elastic-agent keystore would not work because this option does not exist.

1 Like