# Elastic agent upgrade error

**URL:** <https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250>\
**Category:** Kibana\
**Tags:** fleet\
**Created:** [June 27, 2022, 12:51pm UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250 "2022-06-27T12:51:33Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Guncixx](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guncixx/32/87751_2.png) [@Guncixx](https://discuss.elastic.co/u/Guncixx)\
**Post date:** [June 27, 2022, 12:51pm UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250/1 "2022-06-27T12:51:33Z")

</div>

There are similar topics in forum but already closed so opening a new one. After upgrading stack from 8.1.1 to 8.2.3 we are unable to upgrade agents using fleet. When I trigger upgrade function it changes state to upgrading

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/8/28378ef031bba0e64c2ba251b3f10b00472c6475.png)

then, after some time, state changes to unhealthy

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/8/0830f58690ce3078efaa348662c3459580276eda.png)

and then it changes state back to healthy but agent stays at the same version, is not updated.  
When looking at the log files I see that it can't find file for upgrade but is not trying to download it from [artifacts.elastic.co](http://artifacts.elastic.co)

```auto
15:47:35.641
elastic_agent
[elastic_agent][error] 2022-06-27T15:47:35+03:00 - message: Application: [081c640d-fac2-4cd7-b629-51f09907b3c5]: State changed to FAILED: failed verification of agent binary: 2 errors occurred:
	* fetching asc file from '/opt/Elastic/Agent/data/elastic-agent-7f30bb/downloads/elastic-agent-8.2.3-linux-x86_64.tar.gz.asc': open /opt/Elastic/Agent/data/elastic-agent-7f30bb/downloads/elastic-agent-8.2.3-linux-x86_64.tar.gz.asc: no such file or directory
	* open /opt/Elastic/Agent/data/elastic-agent-7f30bb/downloads/elastic-agent-8.2.3-linux-x86_64.tar.gz.sha512: no such file or directory

 - type: 'ERROR' - sub_type: 'FAILED'

```

---

<div class="post-metadata">

**Author:** ![Guncixx](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guncixx/32/87751_2.png) [@Guncixx](https://discuss.elastic.co/u/Guncixx)\
**Post date:** [June 27, 2022, 2:12pm UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250/2 "2022-06-27T14:12:33Z")

</div>

I also did uninstall and fresh install of the agent on one of the hosts and that way everything went without a problem and host is healthy. From that I can only conclude that problem is not in connection between fleet server and agent.

---

<div class="post-metadata">

**Author:** ![JLeysens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jleysens/32/67404_2.png) [@JLeysens](https://discuss.elastic.co/u/JLeysens)\
**Post date:** [June 28, 2022, 11:29am UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250/3 "2022-06-28T11:29:00Z")

</div>

This might be an instance of a known issue. The issue may be addressed once you upgrade your agents to 8.2.0.

> **[Fleet and Elastic Agent 8.1.2 | Fleet and Elastic Agent Guide \[8.1\] | Elastic](https://www.elastic.co/guide/en/fleet/8.1/release-notes-8.1.2.html#known-issues-8.1.2)**

---

<div class="post-metadata">

**Author:** ![Guncixx](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guncixx/32/87751_2.png) [@Guncixx](https://discuss.elastic.co/u/Guncixx)\
**Post date:** [June 29, 2022, 8:27am UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250/4 "2022-06-29T08:27:48Z")

</div>

Does this work only with manual upgrade? We have upgraded our stack to 8.3 and there is now an option in the fleet to choose agent version to upgrade to and same error appears when upgrading to either 8.2 and 8.3

---

<div class="post-metadata">

**Author:** ![JLeysens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jleysens/32/67404_2.png) [@JLeysens](https://discuss.elastic.co/u/JLeysens)\
**Post date:** [June 29, 2022, 8:40am UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250/5 "2022-06-29T08:40:14Z")

</div>

Yeah, from what I can tell a manual upgrade will be required unfortunately.

---

<div class="post-metadata">

**Author:** ![Guncixx](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guncixx/32/87751_2.png) [@Guncixx](https://discuss.elastic.co/u/Guncixx)\
**Post date:** [June 29, 2022, 1:52pm UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250/6 "2022-06-29T13:52:11Z")

</div>

I tried workaround mention here - [[Agent-Upgrade]: For Linux .tar deploy; Agent goes Unhealthy on upgrade with Endpoint Security · Issue #173 · elastic/elastic-agent · GitHub](https://github.com/elastic/elastic-agent/issues/173)

```auto
Linux
Download elastic-agent-8.2.0-linux-x86_64.tar.gz.sha512 and elastic-agent-8.2.0-linux-x86_64.tar.gz.asc from the Elastic Agent download page
Place the files in /opt/Elastic/Agent/data/elastic-agent-xxxxxx/downloads
tested on Debian and RPM distributions
ensure the files are owned by root:root and have permissions of 640
Run the Fleet upgrade process

```

That way we successfully upgraded agents to 8.2, then I initiated upgrade to 8.3, it went through well too but after some time agent went offline. I went on and restarted agent on one of the hosts, after that it changed status to

`Active: activating (auto-restart) (Result: exit-code) since Wed 2022-06-29 08:41:39 CDT; 1min 23s ago`

from log files - `elastic-agent.service: Failed with result 'exit-code'.`

When I execute command elastic-agent status it shows

`/usr/bin/elastic-agent: 2: exec: /opt/Elastic/Agent/elastic-agent: not found`

then looking if such a directory exists I get

`lrwxrwxrwx 1 root root 58 Jun 29 07:07 /opt/Elastic/Agent/elastic-agent -> /opt/Elastic/Agent/data/elastic-agent-1a0f39/elastic-agent`

and lastly looking under `/opt/Elastic/Agent/data/elastic-agent-1a0f39/` I see that there are just logs and vault directories

```auto
ls -l /opt/Elastic/Agent/data/elastic-agent-1a0f39/
total 8
drwx------ 3 root root 4096 Jun 29 08:00 logs
drwxr-x--- 2 root root 4096 Jun 29 07:17 vault

```

If I execute `elastic-agent status` in system where first installed version of an agent was 8.2 and then upgraded to 8.3 I get expected response:

```auto
Status: HEALTHY
Message: (no message)
Applications:
  * endpoint-security (HEALTHY)
                           Protecting with policy {afdd95df-beed-42a7-8233-1bff0ad7ccb7}
  * filebeat_monitoring (HEALTHY)
                           Running
  * metricbeat_monitoring (HEALTHY)
                           Running

```

and listing `elastic-agent-1a0f39` directory there are also more directories then on failing system.

```auto
sudo ls -l /Library/Elastic/Agent/data/elastic-agent-1a0f39
total 99400
drwxr-xr-x 23 root wheel 736 Jun 29 12:18 downloads
-rwxr-xr-x 1 root wheel 50870704 Jun 29 12:18 elastic-agent
drwxr-xr-x 5 root wheel 160 Jun 29 12:20 install
drwx------ 3 root wheel 96 Jun 29 12:18 logs
drwxr-xr-x 3 root wheel 96 Jun 29 12:18 run
-rw------- 1 root wheel 16449 Jun 29 16:31 state.enc

```

---

<div class="post-metadata">

**Author:** ![JLeysens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jleysens/32/67404_2.png) [@JLeysens](https://discuss.elastic.co/u/JLeysens)\
**Post date:** [June 30, 2022, 8:20am UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250/7 "2022-06-30T08:20:06Z")

</div>

That is unfortunate. It is tough to say what went wrong - is there anything else useful in the logs?

Is it possible for you to follow these steps for the agents that failed to upgrade:

1. unenroll agent
2. uninstall agent (this seems to have partially happened already somehow)
3. install newer version
4. re-enroll agent

Referenced from the same GitHub issue you shared:

> <https://github.com/elastic/elastic-agent/issues/173#issuecomment-1117686710>
>
> \*\*Kibana version:\*\* 7.15.0 Snapshot Kibana Cloud environment
> 
> \*\*Host OS and B…rowser version:\*\* VSphere Ubuntu ~and MAC~, All
> 
> \*\*Build details:\*\*
> \`\`\`
> Build: 43937
> Commit: d4c2d0476c622ba2314ab35c3439f1fad4dc0b34
> Artifact Link: https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-7.14.1-linux-x86\_64.tar.gz
> \`\`\`
> 
> \*\*Preconditions:\*\*
> 1. 7.15.0 Snapshot Kibana Cloud environment should be available.
> 2. 7.14.1 released Agent must be installed with Default policy having System and Endpoint Security integration.
> 
> \*\*Steps to reproduce:\*\*
> 1. Login to Kibana environment.
> 2. Trigger Agent upgrade from Fleet UI for \`7.14.1 release\` agent.
> 3. Observe agent went \`Unhealthy\` after upgrade.
> 
> \*\*Debug level Logs:\*\*
> \[logs.zip\](https://github.com/elastic/beats/files/7114909/logs.zip)
> \[endpoint-000000.zip\](https://github.com/elastic/beats/files/7114911/endpoint-000000.zip)
> 
> \*\*Note:\*\*
> \- This issue is observed on Vsphere machines name: linux \`qa-ubuntu20.04-desktop\` ~and mac \`qa-mac-bigsur-11.0.1-release-nosip-clone-base\`~
> \- This issue is not observed on AWS-Ubuntu 20
> 
> \*\*Expected Result:\*\*
> \`7.14.1\` Ubuntu \`.tar\` agent should upgrade to \`7.15.0\` with Endpoint Security and should remain \`Healthy\`.
> 
> \*\*Screenshots:\*\*
> !\[5\](https://user-images.githubusercontent.com/77374876/132196219-fb1c975a-8e22-412f-baed-4b2deaa8591e.JPG)
> !\[6\](https://user-images.githubusercontent.com/77374876/132196238-87fdeb7e-a0f5-4d1a-8568-c5637cb18325.JPG)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 28, 2022, 8:20am UTC](https://discuss.elastic.co/t/elastic-agent-upgrade-error/308250/8 "2022-07-28T08:20:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
