How to use elastic-agent docker image?

[klausagnoletti]

Hi

I've been wanting to run elastic-agent as a docker container for quite some time. So I was quite ecstatic when I found out it was available for download at beats/elastic-agent | Docker @ Elastic. However there's no documentation available anywhere (at least not where I can find it). So I have been doing trial-and-error to make it run via docker-compose. And I can almost make it work. Only problem is that it doesn't really collect any data. Which sucks. So if anybody could provide me with any help on getting it to run, I will be extremely grateful

Thanks!

/klaus

[mostlyjason (Jason Skowronski)]

We haven't written up the documentation yet, but that is our to do list. Here's what I use for my Docker compose:

services:
  elastic-agent:
    image: elastic/elastic-agent:7.11.0
    restart: unless-stopped
    hostname: 'docker-agent'
    environment:
      FLEET_ENROLL: 1
      FLEET_ENROLL_INSECURE: 1
      KIBANA_HOST: "http://kibana:5601"
      FLEET_ENROLLMENT_TOKEN: "my_token"
    user: root
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro

You can find more environment variables here beats/container.go at 23efd31aeaa6143a419c61c633f8c21d8a599264 · elastic/beats · GitHub

I have mine running in an agent policy with the system integration enabled, which collect system metrics and logs. Seems to be working for me. If you're having trouble perhaps check the elastic agent logs inside the container?

[klausagnoletti]

Hey Jason and thanks for getting back. This is what I have in docker-compose (as part of a file that starts the entire stack and more):

elastic-agent:
        hostname: ${HOSTNAME}
        image: docker.elastic.co/beats/elastic-agent:${ELK_VERSION}
        restart: always
        user: root
        environment:
            # FLEET_CONFIG_ID - config related to new token [defaul]
            # FLEET_ENROLLMENT_TOKEN - existing enrollment token to be used for enroll
            FLEET_ENROLLMENT_TOKEN: "xxxx"
            # FLEET_ENROLL - if set to 1 enroll will be performed
            #FLEET_ENROLL: "1"
            # FLEET_ENROLL_INSECURE - if set to 1, agent will enroll with fleet using --insecure flag
            # FLEET_SETUP - if  set to 1 fleet setup will be performed
            #FLEET_SETUP: "1"
            # FLEET_TOKEN_NAME - token name for a token to be created
            # KIBANA_HOST - actual kibana host [http://localhost:5601]
            KIBANA_HOST: https://${KIBANA_HOST}:${KIBANA_PORT}
            # KIBANA_USERNAME - username for accessing kibana API [elastic]
            KIBANA_USERNAME: ${ELASTIC_USERNAME}
            # KIBANA_PASSWORD - password for accessing kibana API [changeme]
            KIBANA_PASSWORD: ${ELASTIC_PASSWORD}
            ELASTICSEARCH_USERNAME: ${ELASTIC_USERNAME}
            ELASTICSEARCH_PASSWORD: ${ELASTIC_PASSWORD}
            ELASTICSEARCH_HOSTS: https://${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}
        labels:
            com.centurylinklabs.watchtower.enable: "false"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
            - /mnt/cache/appdata/elastic-agent/config/elastic-agent.yml:/usr/share/elastic-agent/elastic-agent.yml:ro
            - /sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro
            - /proc:/hostfs/proc:ro
            - /:/hostfs:ro
            - /var/log:/var/log:rw
        networks:
            macvlan-lan:
                aliases:
                    - ${HOSTNAME}
                ipv4_address: xxxxx

I find it a bit cumbersome to use in terms of enrolling. I found that just enable enrolling made the agent enroll again and again, so for me it worked to start it once with enrolling disabled, disable and rebuild the container to prevent that. But still the agent is marked as unhealthy and doesn't send any data. I haven't figured out how to extract the config from the beats agents running (filebeat and metricbeat) or even how to manipulate the config. So in that sense I am a bit stuck.

Thanks for link to the environment variables, I'll check it out.

/k

[klausagnoletti]

Allright, so after a bit of testing I know more about what's going on. I am able to replicate your docker-compose with having a healthy elastic-agent collecting system metrics. But when I apply my default Linux polixy doing auditd, endpoint security and what not, things go haywire with the agent being marked as unhealthy.

So I guess there are limitations as to which policies work and which don't? Or are there extra steps that needs to be taken for the other policies to work? For instance if I want to do stack monitoring with metricbeat - or log file auditing with filebeat? Both things are possible using standalone beats (which I used to do). I am running the entire stack in docker containers on the same physical server (which is why I can't run elastic-agent natively on the host server.

Thanks!

/k