# Elastic Agent ECK 7.17.0 Work Around doesn't Work Correctly

**URL:** https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335
**Category:** Elastic Cloud on Kubernetes (ECK)
**Tags:** docker
**Created:** [February 4, 2022, 6:35pm UTC](https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335 "2022-02-04T18:35:31Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![BenB196](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benb196/32/83401_2.png) [@BenB196](https://discuss.elastic.co/u/BenB196)
#### Post date: [February 4, 2022, 6:35pm UTC](https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335/1 "2022-02-04T18:35:31Z")

</div>

Hi All,

There is known issue with Elastic Agent managed by ECK where it doesn't work for 7.17.0.

There is a provided workaround: [https://github.com/elastic/cloud-on-k8s/issues/5323](https://github.com/elastic/cloud-on-k8s/issues/5323) (and [https://github.com/elastic/cloud-on-k8s/pull/5326/files](https://github.com/elastic/cloud-on-k8s/pull/5326/files))

But this doesn't appear to work correctly. Example

`7.16.3` - Startup Logs

```auto
2022-02-04T18:32:41.557Z INFO application/application.go:67 Detecting execution mode
2022-02-04T18:32:41.557Z INFO application/application.go:76 Agent is managed locally
2022-02-04T18:32:41.557Z INFO capabilities/capabilities.go:59 capabilities file not found in /usr/share/elastic-agent/state/capabilities.yml
2022-02-04T18:32:43.351Z INFO [composable.providers.docker] docker/docker.go:43 Docker provider skipped, unable to connect: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
2022-02-04T18:32:43.352Z INFO [api] api/server.go:62 Starting stats endpoint
2022-02-04T18:32:43.352Z INFO application/local_mode.go:168 Agent is starting
2022-02-04T18:32:43.352Z INFO [api] api/server.go:64 Metrics endpoint listening on: /usr/share/elastic-agent/state/data/tmp/elastic-agent.sock (configured: unix:///usr/share/elastic-agent/state/data/tmp/elastic-agent.sock)
2022-02-04T18:32:43.353Z INFO application/local_mode.go:178 Agent is stopped
2022-02-04T18:32:43.353Z INFO application/periodic.go:79 Configuration changes detected
2022-02-04T18:32:43.359Z INFO stateresolver/stateresolver.go:48 New State ID is I1pGg08w
2022-02-04T18:32:43.359Z INFO stateresolver/stateresolver.go:49 Converging state requires execution of 2 step(s)
2022-02-04T18:32:51.837Z INFO log/reporter.go:40 2022-02-04T18:32:51Z - message: Application: heartbeat--7.16.3[e07f11bb-01dc-4270-91a6-ee08f369505c]: State changed to STARTING: Starting - type: 'STATE' - sub_type: 'STARTING'
2022-02-04T18:32:54.542Z INFO log/reporter.go:40 2022-02-04T18:32:54Z - message: Application: heartbeat--7.16.3[e07f11bb-01dc-4270-91a6-ee08f369505c]: State changed to RUNNING: Running - type: 'STATE' - sub_type: 'RUNNING'
2022-02-04T18:33:08.843Z INFO log/reporter.go:40 2022-02-04T18:33:08Z - message: Application: filebeat--7.16.3--36643631373035623733363936343635[e07f11bb-01dc-4270-91a6-ee08f369505c]: State changed to STARTING: Starting - type: 'STATE' - sub_type: 'STARTING'
2022-02-04T18:33:18.241Z INFO log/reporter.go:40 2022-02-04T18:33:18Z - message: Application: filebeat--7.16.3--36643631373035623733363936343635[e07f11bb-01dc-4270-91a6-ee08f369505c]: State changed to RUNNING: Running - type: 'STATE' - sub_type: 'RUNNING'
2022-02-04T18:33:29.141Z INFO log/reporter.go:40 2022-02-04T18:33:29Z - message: Application: metricbeat--7.16.3--36643631373035623733363936343635[e07f11bb-01dc-4270-91a6-ee08f369505c]: State changed to STARTING: Starting - type: 'STATE' - sub_type: 'STARTING'
2022-02-04T18:33:29.143Z INFO stateresolver/stateresolver.go:66 Updating internal state
2022-02-04T18:33:29.143Z INFO application/periodic.go:101 No configuration change
2022-02-04T18:33:29.146Z INFO stateresolver/stateresolver.go:48 New State ID is I1pGg08w
2022-02-04T18:33:29.146Z INFO stateresolver/stateresolver.go:49 Converging state requires execution of 0 step(s)
2022-02-04T18:33:29.146Z INFO stateresolver/stateresolver.go:66 Updating internal state
2022-02-04T18:33:34.951Z INFO log/reporter.go:40 2022-02-04T18:33:34Z - message: Application: metricbeat--7.16.3--36643631373035623733363936343635[e07f11bb-01dc-4270-91a6-ee08f369505c]: State changed to RUNNING: Running - type: 'STATE' - sub_type: 'RUNNING'

```

`7.17.0`

```auto
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
2022-02-04T18:08:20.944Z INFO application/application.go:67 Detecting execution mode
2022-02-04T18:08:20.944Z INFO application/application.go:76 Agent is managed locally
2022-02-04T18:08:20.944Z INFO capabilities/capabilities.go:59 capabilities file not found in /usr/share/elastic-agent/state/capabilities.yml
2022-02-04T18:08:21.753Z INFO [composable.providers.docker] docker/docker.go:43 Docker provider skipped, unable to connect: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
2022-02-04T18:08:21.754Z INFO [api] api/server.go:62 Starting stats endpoint
2022-02-04T18:08:21.754Z INFO application/local_mode.go:168 Agent is starting
2022-02-04T18:08:21.754Z INFO [api] api/server.go:64 Metrics endpoint listening on: /usr/share/elastic-agent/state/data/tmp/elastic-agent.sock (configured: unix:///usr/share/elastic-agent/state/data/tmp/elastic-agent.sock)
2022-02-04T18:08:21.754Z INFO application/local_mode.go:178 Agent is stopped
2022-02-04T18:08:21.755Z INFO application/periodic.go:79 Configuration changes detected
2022-02-04T18:08:21.762Z INFO stateresolver/stateresolver.go:48 New State ID is 3xwWCltv
2022-02-04T18:08:21.762Z INFO stateresolver/stateresolver.go:49 Converging state requires execution of 2 step(s)
2022-02-04T18:08:35.835Z INFO log/reporter.go:40 2022-02-04T18:08:35Z - message: Application: metricbeat--7.17.0[ab5ebc20-bd2b-4f17-ad22-11dee796e22a]: State changed to STARTING: Starting - type: 'STATE' - sub_type: 'STARTING'
2022-02-04T18:08:43.538Z INFO log/reporter.go:40 2022-02-04T18:08:43Z - message: Application: metricbeat--7.17.0[ab5ebc20-bd2b-4f17-ad22-11dee796e22a]: State changed to RUNNING: Running - type: 'STATE' - sub_type: 'RUNNING'
2022-02-04T18:08:51.036Z INFO log/reporter.go:40 2022-02-04T18:08:51Z - message: Application: filebeat--7.17.0--36643631373035623733363936343635[ab5ebc20-bd2b-4f17-ad22-11dee796e22a]: State changed to STARTING: Starting - type: 'STATE' - sub_type: 'STARTING'
2022-02-04T18:08:55.044Z INFO operation/operator.go:284 operation 'operation-install' skipped for metricbeat.7.17.0
2022-02-04T18:08:56.039Z INFO log/reporter.go:40 2022-02-04T18:08:56Z - message: Application: metricbeat--7.17.0--36643631373035623733363936343635[ab5ebc20-bd2b-4f17-ad22-11dee796e22a]: State changed to STARTING: Starting - type: 'STATE' - sub_type: 'STARTING'
2022-02-04T18:08:56.041Z INFO stateresolver/stateresolver.go:66 Updating internal state
2022-02-04T18:08:56.041Z INFO application/periodic.go:101 No configuration change
2022-02-04T18:08:56.137Z INFO stateresolver/stateresolver.go:48 New State ID is 3xwWCltv
2022-02-04T18:08:56.138Z INFO stateresolver/stateresolver.go:49 Converging state requires execution of 0 step(s)
2022-02-04T18:08:56.138Z INFO stateresolver/stateresolver.go:66 Updating internal state
2022-02-04T18:08:58.939Z INFO log/reporter.go:40 2022-02-04T18:08:58Z - message: Application: filebeat--7.17.0--36643631373035623733363936343635[ab5ebc20-bd2b-4f17-ad22-11dee796e22a]: State changed to RUNNING: Running - type: 'STATE' - sub_type: 'RUNNING'
2022-02-04T18:09:03.241Z INFO log/reporter.go:40 2022-02-04T18:09:03Z - message: Application: metricbeat--7.17.0--36643631373035623733363936343635[ab5ebc20-bd2b-4f17-ad22-11dee796e22a]: State changed to RUNNING: Running - type: 'STATE' - sub_type: 'RUNNING'

```

The agent for 7.17.0 uses a completely different config than it does on 7.16.3

---

<div class="post-metadata">

### Author: ![BenB196](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benb196/32/83401_2.png) [@BenB196](https://discuss.elastic.co/u/BenB196)
#### Post date: [February 4, 2022, 6:38pm UTC](https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335/2 "2022-02-04T18:38:14Z")

</div>

In `7.16.3` the proper config is loaded (note: Heartbeat in logs)

In `7.17.0` a default config is loaded (note: no Heartbeat in logs)

Config of Agent:

`7.16.3`

```auto
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: elastic-agent-test
  namespace: elastic-prod
spec:
  configRef:
    secretName: elastic-agent-test-config
  deployment:
    podTemplate:
      metadata:
        creationTimestamp: null
      spec:
        containers:
        - env:
          - name: BEAT_SETUID_AS
            value: root
          name: agent
          resources: {}
          securityContext:
            allowPrivilegeEscalation: false
            runAsUser: 0
    strategy: {}
  elasticsearchRefs:
  - name: es-prod
    namespace: elastic-prod
  fleetServerRef:
    name: ""
  http:
    service:
      metadata: {}
      spec: {}
    tls:
      certificate: {}
  kibanaRef:
    name: ""
  version: 7.16.3

```

`7.17.0`

```auto
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: elastic-agent-test
  namespace: elastic-prod
spec:
  configRef:
    secretName: elastic-agent-test-config
  deployment:
    podTemplate:
      metadata:
        creationTimestamp: null
      spec:
        containers:
        - command:
          - bash
          - -c 
          - |
            #!/usr/bin/env bash
            set -e
            if [[-f /mnt/elastic-internal/elasticsearch-association/<agent-ns>/<es-name>/certs/ca.crt]]; then
              cp /mnt/elastic-internal/elasticsearch-association/<agent-ns>/<es-name>/certs/ca.crt /usr/local/share/ca-certificates
              update-ca-certificates
            fi
            /usr/bin/tini -- /usr/local/bin/docker-entrypoint -e
          env:
          - name: BEAT_SETUID_AS
            value: root
          name: agent
          resources: {}
          securityContext:
            allowPrivilegeEscalation: false
            runAsUser: 0
    strategy: {}
  elasticsearchRefs:
  - name: es-prod
    namespace: elastic-prod
  fleetServerRef:
    name: ""
  http:
    service:
      metadata: {}
      spec: {}
    tls:
      certificate: {}
  kibanaRef:
    name: ""
  version: 7.17.0

```

Note: I did replace both instances of `<agent-ns>/<es-name>` with the correct path.

---

<div class="post-metadata">

### Author: ![pebrc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pebrc/32/101790_2.png) [@pebrc](https://discuss.elastic.co/u/pebrc)
#### Post date: [February 5, 2022, 4:05pm UTC](https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335/3 "2022-02-05T16:05:14Z")

</div>

I tried to reproduce your problem but was not able to. I was able to deploy Elastic Agent/Fleet Server and installed a simple uptime check via the Kibana UI with success. Can you maybe share more about the Uptime/Heartbeat configuration you are trying to deploy?

---

<div class="post-metadata">

### Author: ![BenB196](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benb196/32/83401_2.png) [@BenB196](https://discuss.elastic.co/u/BenB196)
#### Post date: [February 5, 2022, 11:46pm UTC](https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335/4 "2022-02-05T23:46:25Z")

</div>

Hi @pebrc,

Here is the full Kubernetes config for the agent:

```auto
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: elastic-agent-test
  namespace: elastic-prod
spec:
  configRef:
    secretName: elastic-agent-test-config
  deployment:
    podTemplate:
      metadata:
        creationTimestamp: null
      spec:
        containers:
        - env:
          - name: BEAT_SETUID_AS
            value: root
          name: agent
          resources: {}
          securityContext:
            allowPrivilegeEscalation: false
            runAsUser: 0
    strategy: {}
  elasticsearchRefs:
  - name: es-prod
    namespace: elastic-prod
  fleetServerRef:
    name: ""
  http:
    service:
      metadata: {}
      spec: {}
    tls:
      certificate: {}
  kibanaRef:
    name: ""
  version: 7.16.3

```

Secret with the Agent config

```auto
apiVersion: v1
stringData:
  agent.yml: |- 
    inputs:
      - id: ping-test
        name: ping-test
        revision: 2
        type: synthetics/icmp
        use_output: default
        meta:
          package:
            name: synthetics
            version: 0.5.0
        data_stream:
          namespace: dev
        streams:
          - id: ping-test
            name: ping-test
            type: icmp
            data_stream:
              dataset: icmp
              type: synthetics
            schedule: '@every 10s'
            wait: 1s
            hosts: 
            - example.com
            - example2.com
            timeout: 8s
            tags:
              - test
kind: Secret
metadata:
  name: elastic-agent-test-config
  namespace: elastic-prod
type: Opaque

```

The above 2 configs work with 7.16.3, but do not work on 7.17.0 with the workaround applied.

---

<div class="post-metadata">

### Author: ![BenB196](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benb196/32/83401_2.png) [@BenB196](https://discuss.elastic.co/u/BenB196)
#### Post date: [February 13, 2022, 3:22pm UTC](https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335/5 "2022-02-13T15:22:58Z")

</div>

This actually turns out to be a misunderstanding of the issue + workaround.

I had initially thought that all instances of Elastic Agent on ECK needed this. However, this is incorrect, only the Fleet managed ones needed the workaround.

A question though regarding the work around:

In `7.16.3` or earlier, the Fleet managed command started with:

```auto
/usr/bin/env

```

However, the workaround for `7.17.0` omits this first line.

Is this intended, or is this line not needed?

---

<div class="post-metadata">

### Author: ![pebrc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pebrc/32/101790_2.png) [@pebrc](https://discuss.elastic.co/u/pebrc)
#### Post date: [February 14, 2022, 9:58am UTC](https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335/6 "2022-02-14T09:58:50Z")

</div>

> [@BenB196](#):
>
> Is this intended, or is this line not needed?

I think this was just an oversight.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 14, 2022, 9:59am UTC](https://discuss.elastic.co/t/elastic-agent-eck-7-17-0-work-around-doesnt-work-correctly/296335/7 "2022-03-14T09:59:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.