Elastic Agent ignores the Agent Binary Download location setting

[username11]

Hi. This is basically a repeat of this thread, which did not get any replies. Right now I am testing the Elastic Agents in a test environment, which does not have direct internet access (and neither will my live environment). I've deployed a local Artifacts repo, which I've confirmed to be working, and set it as the default location in Agent Binary Download option in Fleet GUI.

Inspecting the agent's policy confirms that the setting is applied:

[root@agent2 logs]# elastic-agent inspect | head -n 15
agent:
  download:
    source_uri: https://elastic-repo.local/downloads/
  monitoring:
    enabled: true
    logs: true
    metrics: true
    namespace: default
    use_output: default

However, attempting to upgrade shows that the agent checks the local package cache, then goes straight to the Elastic Repo:

{"log.level":"info","@timestamp":"2022-12-01T17:43:01.959Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2022-12-01T12:43:01-05:00 - message: Application: [96d57c8b-ade3-4206-80d3-b33c21ebfabd]: State changed to UPDATING: Update to version '8.5.0' started - type: 'STATE' - sub_type: 'UPDATING'","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-12-01T17:43:03.511Z","log.origin":{"file.name":"log/reporter.go","file.line":36},"message":"2022-12-01T12:43:03-05:00 - message: Application: [96d57c8b-ade3-4206-80d3-b33c21ebfabd]: State changed to FAILED: failed upgrade of agent binary: 2 errors occurred:\n\t* package '/opt/Elastic/Agent/data/elastic-agent-6e9dd4/downloads/elastic-agent-8.5.0-linux-x86_64.tar.gz' not found: open /opt/Elastic/Agent/data/elastic-agent-6e9dd4/downloads/elastic-agent-8.5.0-linux-x86_64.tar.gz: no such file or directory\n\t* call to 'https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.5.0-linux-x86_64.tar.gz'

Has anyone else encountered this? Is the local repo supported at all, or can this somehow be fixed?

[username11]

One thing I should mention is that I also changed the default Elastic Repo setting to "https://disabled.artifacts.elastic.co/downloads/", and that URL was not used by the agent, so it's not a case of it using the default setting - more like the URL being hardcoded in the agent.

[MichelLaterman]

Hello,

The issue you are running into appears to be: https://github.com/elastic/elastic-agent/pull/1252 which has been solved in our latest release.

Unfortunately, that makes upgrading from a broken version to a fixed version tricky.
What version are you running?

To resolve this upgrade, you may want to manually download the upgraded agent version + checksum into the install downloads dir, or alter the DNS record to point to your repo instead of ours for (for one upgrade)

[username11]

Thanks for the answer. I was running Elastic Agent v8.0.1 + Fleet v.8.5.2. Upgrading shouldn't be a problem for me, as this is still a test environment.

[username11]

Follow-up: upgrading the agent version has indeed solved the problem. Thanks again.