Fleet agent upgrade fails 8.1.2

Elastic Stack Beats
1

After the stack upgrade to the latest 8.1.3 the agents are unable to upgrade

 error: failed verification of agent binary: 2 errors occurred:\n\t* fetching asc file from 'C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.asc': open C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.asc: The system cannot find the file specified.\n\t* open C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.sha512: The system cannot find the file specified.\n\n","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-21T21:53:19.122Z","log.origin":{"file.name":"status/reporter.go","file.line":236},"message":"Elastic Agent status changed to: 'error'","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-21T21:55:38.709Z","log.origin":{"file.name":"fleet/fleet_gateway.go","file.line":206},"message":"Could not communicate with fleet-server Checking API will retry, error: fail to checkin to fleet-server: Post \"https://192.168.131.155:8220/api/fleet/agents/17a97d7d-5ce3-4004-9f0f-f0c7c889efc6/checkin?\": EOF","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-21T21:56:48.808Z","log.origin":{"file.name":"fleet/fleet_gateway.go","file.line":206},"message":"Could not communicate with fleet-server Checking API will retry, error: fail to checkin to fleet-server: Post \"https://192.168.131.155:8220/api/fleet/agents/17a97d7d-5ce3-4004-9f0f-f0c7c889efc6/checkin?\": dial tcp 192.168.131.155:8220: connectex: No connection could be made because the target machine actively refused it.","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-21T22:02:00.098Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2022-04-22T08:02:00+10:00 - message: Application: [17a97d7d-5ce3-4004-9f0f-f0c7c889efc6]: State changed to UPDATING: Update to version '8.1.3' started - type: 'STATE' - sub_type: 'UPDATING'","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-21T22:03:03.238Z","log.origin":{"file.name":"log/reporter.go","file.line":36},"message":"2022-04-22T08:03:03+10:00 - message: Application: [17a97d7d-5ce3-4004-9f0f-f0c7c889efc6]: State changed to FAILED: failed verification of agent binary: 2 errors occurred:\n\t* fetching asc file from 'C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.asc': open C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.asc: The system cannot find the file specified.\n\t* open C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.sha512: The system cannot find the file specified.\n\n - type: 'ERROR' - sub_type: 'FAILED'","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2022-04-21T22:03:03.238Z","log.origin":{"file.name":"fleet/fleet_gateway.go","file.line":181},"message":"failed to dispatch actions, error: failed verification of agent binary: 2 errors occurred:\n\t* fetching asc file from 'C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.asc': open C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.asc: The system cannot find the file specified.\n\t* open C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-6118f2\\downloads\\elastic-agent-8.1.3-windows-x86_64.zip.sha512: The system cannot find the file specified.\n\n","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-04-21T22:07:49.028Z","log.origin":{"file.name":"status/reporter.go","file.line":236},"message":"Elastic Agent status changed to: 'online'","ecs.version":"1.6.0"}

Same thing happened with 8.1.1 to 8.1.2

Am I missing something, do I need to place the zip file in the downloads location, I thought this was meant to be automated

Thanks

2

I see the same thing, the agents fail to upgrade, Windows and Linux and we have to upgrade manually.

3

You're probably running into this known issue: Fleet and Elastic Agent 8.1.2 | Fleet and Elastic Agent Guide [8.1] | Elastic, in theory is fixed in 8.1.3 (but idk if you can really test until 8.2.0 gets released).

4

from 8.1.1 to 8.1.2 to 8.1.3 same issue in practise ... Always update when minor issues are released and always have issue with linux distro packages, either the verification, missing the download package or something else breaking.

Fleet Managed Elastic-Agents are really not GA worthy releases... still cant live up to Beats.

5

Yeah, I'm hoping that the 8.1.3 fix actually fixes the issue, as the upgrades on Linux have always been a bit flaky via the UI, and not being able to uninstall (& unregister) to reinstall/upgrade has always been a bit annoying.

6

Agreed, I cannot push forward a solution to replace the current Beats setup, since every version release brings a new issue to kill any wide-scale deployment hopes in Production.

7

Thanks for your input guys, I'm seeing the same thing so far.
Going to stick with beats for now but keep a test server and monitor the changes

9

@BenB196 Shame the link is not working anymore. Would like to see more about this issue.

@VamPikmin I found a workaround, but it is far from ideal. It is possible to upgrade from 8.1.2 to 8.1.3. I got it working by unenroll Elastic Agent in Fleet and uninstalling Elastic Agent from the client with 8.1.2 version installed and then download 8.1.3 and install it.

1 Like
10

@JimG Yep, that's what I've been doing too. Thanks for your input

11

@VamPikmin Allright, just saw a new version of elastic came out (8.2.0) , maybe the function will work there.

I got a homelab where i'll be testning if this issue exist when i upgrade from 8.1.2 --> 8.1.3 if i got the same issue, i'll try upgrading to 8.2.0 and see it is possible to upgrade elastic agents from fleet.

12

@JimG Upgraded manually from 8.1.2 => 8.1.3 (same issue as before)

8.1.3 => 8.2.0 Worked automatically, so it's been fixed. Thanks Elastic

1 Like
13

@VamPikmin Well its kinda working for me as well. The Elastic Agent that is upgraded to 8.1.3 will upgrade without any problems to 8.2.0.

but the 8.1.2 --> 8.2.0 post the same error.

1 Like
14

@JimG

Yep, that's what I've found too. Let's see what happens next version

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.