Impossible to uninstall Elastic Endpoint

Elastic Stack Elastic Agent
1

Hello,
I've uninstalled the elastic agent on a windows machine successfully, the uninstall command runs successfully but, the Elastic Endpoint Folder is still present.

image
image873×375 26.5 KB

without success; it runs but the folder is still present.

  • Tried to reinstall the elastic agent but gives me the error

image

How I can definitively remove the folder C:\Program Files\Elastic\Endpoint ?

Thanks in advance for the help.

2

Hi,

Run the following command to stop the Elastic Endpoint service:

sc stop "Elastic Endpoint"

Once the service is stopped, try to delete the folder again.

Regards

3

Hi,

I'm Endpoint developer.

We don't even know what version is installed. However it shouldn't matter when following the uninstall guide, unless it's a very recent Endpoint with Tamper Protection turned ON.

Could you send me a console log from executing

cd %TEMP%
copy "c:\Program Files\Elastic\Endpoint\elastic-endpoint.exe" elastic-endpoint.exe
.\elastic-endpoint.exe version
.\elastic-endpoint.exe uninstall --log stdout --log-level trace
del .\elastic-endpoint.exe
2 Likes
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

6

Hi, thanks a lot for the help.
Here the output of the commands

Version

version: 8.9.1, compiled: Tue Aug 8 15:00:00 2023, branch: 8.9, commit: 1d13054232606508c9e4c0a53380f2e20158b9f9

Unprotect command .\elastic-endpoint.exe unprotect no output received

Uninstall command with trace (output too heavy, here an extract but all the output is pretty similar (

2024-03-14 09:29:42: info: Main.cpp:287 Executing uninstall
2024-03-14 09:29:42: debug: Service.cpp:817 PPL is supported. This process is unprotected. (TrustLevelSid: absent)
2024-03-14 09:29:42: error: Service.cpp:329 OpenServiceW(ElasticEndpointDriver) failed with error 1060
2024-03-14 09:29:42: trace: Service.cpp:330 Function returned error status (Failure in an external software component) because of system status (1060/The specified service does not exist as an installed service.)
2024-03-14 09:29:42: trace: Service.cpp:389 Function returned error status (Failure in an external software component)
2024-03-14 09:29:42: trace: File.cpp:1060 Function returned error status (Failure in an external software component) because of system status (2/The system cannot find the file specified.)
2024-03-14 09:29:42: trace: File.cpp:1145 Function returned error status (Failure in an external software component)
2024-03-14 09:29:42: trace: File.cpp:244 Function returned error status (Failure in an external software component)
2024-03-14 09:29:42: trace: Util.cpp:2081 Function returned error status (Failure in an external software component)
2024-03-14 09:29:42: error: Service.cpp:178 OpenServiceW(ElasticEndpoint) failed with error 1060
2024-03-14 09:29:42: trace: Service.cpp:179 Function returned error status (Failure in an external software component) because of system status (1060/The specified service does not exist as an installed service.)
2024-03-14 09:29:42: error: Util.cpp:1383 Endpoint service was unable to be deleted or scheduled for deletion.
2024-03-14 09:29:42: error: Service.cpp:178 OpenServiceW(ElasticEndpointDriver) failed with error 1060
2024-03-14 09:29:42: trace: Service.cpp:179 Function returned error status (Failure in an external software component) because of system status (1060/The specified service does not exist as an installed service.)
2024-03-14 09:29:42: error: Util.cpp:1401 Endpoint driver service was unable to be deleted or scheduled for deletion.
2024-03-14 09:29:42: error: Service.cpp:329 OpenServiceW(ElasticELAMDriver) failed with error 1060
2024-03-14 09:29:42: trace: Service.cpp:330 Function returned error status (Failure in an external software component) because of system status (1060/The specified service does not exist as an installed service.)
2024-03-14 09:29:42: trace: Service.cpp:389 Function returned error status (Failure in an external software component)
2024-03-14 09:29:42: error: Service.cpp:178 OpenServiceW(ElasticELAMDriver) failed with error 1060
2024-03-14 09:29:42: trace: Service.cpp:179 Function returned error status (Failure in an external software component) because of system status (1060/The specified service does not exist as an installed service.)
2024-03-14 09:29:42: error: Util.cpp:1427 ELAM driver service was unable to be deleted or scheduled for deletion.
2024-03-14 09:29:42: trace: Util.cpp:396 Function returned error status (Failed to delete registry key)
2024-03-14 09:29:42: trace: Util.cpp:412 Function returned error status (Failed to delete registry key)
2024-03-14 09:29:42: debug: File.cpp:501 Removing [C:\Program Files\Elastic\Endpoint\elastic-endpoint.yaml]
2024-03-14 09:29:42: trace: File.cpp:508 Function returned error status (I/O error) because of system status (5/Access is denied.)
2024-03-14 09:29:42: info: File.cpp:528 Attempted deletion failed, failed to reset file attributes for C:\Program Files\Elastic\Endpoint\elastic-endpoint.yaml
2024-03-14 09:29:42: debug: File.cpp:501 Removing [C:\Program Files\Elastic\Endpoint\SecurityProductInformation.ini]
2024-03-14 09:29:42: trace: File.cpp:508 Function returned error status (I/O error) because of system status (5/Access is denied.)
2024-03-14 09:31:22: info: File.cpp:528 Attempted deletion failed, failed to reset file attributes for C:\Program Files\Elastic\Endpoint\NOTICE.txt
2024-03-14 09:31:22: trace: PlatformFile.cpp:1011 Function returned error status (Failure in an external software component)
2024-03-14 09:31:22: trace: PlatformFile.cpp:1009 Function returned error status (Invalid parameter)
2024-03-14 09:31:22: trace: PlatformFile.cpp:1009 Function returned error status (Invalid parameter)
2024-03-14 09:31:22: trace: File.cpp:703 Function returned error status (I/O error)
2024-03-14 09:31:22: trace: File.cpp:703 Function returned error status (I/O error)
2024-03-14 09:31:22: trace: Util.cpp:647 Function returned error status (Not found)
2024-03-14 09:31:22: trace: Util.cpp:836 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:606 Function returned error status (Invalid parameter)
2024-03-14 09:31:22: trace: PlatformFile.cpp:965 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:986 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:536 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:984 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:536 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:984 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:536 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:984 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:536 Function returned error status (Not found)
2024-03-14 09:31:22: trace: PlatformFile.cpp:984 Function returned error status (Not found)

I have the console log saved on a txt file; how I can share it? Pastebin is ok?

Thanks again for the assistance.

Sincerely

7

I'll send you PM with an URL where you can upload the log(s).

I'm not sure how your machine ended up in this state.

I would like to see the full log from the uninstall. In addition I'd like to ask for output of:

cmd /c sc query "ElasticEndpoint"

Could you download a Sysinternals utility https://live.sysinternals.com/accesschk64.exe and also provide me output of this command:

accesschk64.exe -accepteula -l -v "c:\Program Files\Elastic\Endpoint"

Thanks

8

Hi,
This machine ended up in this state when I was migrating the agent to a new elastic server installation; instead of uninstall and reinstall, I've unenrolled it and enrolled again into the new server installation.

The strange thing ( I've only noticed it now ) is that the service does not exists, also if I try to install it again : in fact the sc query command gives "Service does not exists".

The problem is only on deleting the C:\Program Files\Elastic\Endpoint folder and the files inside it.

I've uploaded a zip with two files :

  • uninstall.txt -> the complete logs for uninstall command
  • accesschk.txt -> logs for the accesschk64 output

Tell me if you need something else.

Thanks for the assistance.

9

Hi, it would be hard to troubleshoot it further. The accesschk log confirms that you're using the right elastic-endpoint.exe.

At this point the simplest solution would be to boot the machine from Linux (bootable CD or pendrive) to delete the entire Endpoint directory bypassing Windows.

1 Like
10

Hi,
Thanks for your analysis, since it is a Domain Controller (on a VM) is not too easy perform this solution.

Anyway, thanks again for your time, if there are no other alternatives to unlock this folder, I will try your solution the next available maintenance window on the server.

Bye, have a good day!

11

Thanks for the patience. I'm glad to hear that you'll eventually be able to fix it.

We are already aware of this problem. The file system may remain in an inconsistent state due to sudden power off, shutdown, etc. So far, the steps we have already tried here have been sufficient to recover from the situation otherwise. In this case, there NTFS is corrupted in a unique way for which we would most likely have to develop a one-time recovery tool. Linux doesn't care about SACL/DACL security on NTFS, so it will let you remove it cleanly.

Sorry again for the trouble. We're working on a new version of the files protection which will use different mechanisms to achieve the same goal.

have a nice day, and hope you sill like using Elastic Defend

1 Like
12

Hi Lesio,

We deployed Elastic Defend on Windows with Tamper Protection turned ON, then we tried to upgrade the agent from the fleet but it did not work and was stuck on updating, then we assigned the agent to another policy without Elastic Defend to make the upgrade working.
After the upgrade completed we added Elastic Defend but the agent becomes unhealthy because old endpoint.
We would like to force uninstall the old endpoint machine, if you have a way without using uninstall token because we lost it I think due to all changes made.

Thanks & Best Regards,

Hamid Allaoui

13

Hi Hamid Allaoui,

Obviously the role of uninstall protection is to make it immune to simple unauthorized "force remove" attempts.

To clarify, are we talking about the same environment troubleshooted in this thread or you've just found this thread when troubleshooting your environment?

Assuming it's a new case, I'd suggest to try first the straightforward way of uninstalling tamper protected Endpoint.

Go to Fleet -> Uinstall tokens, grab the uninstall token

image
image1824×603 41.4 KB

Then execute on the target computer from elevated command prompt

"C:\Program Files\Elastic\Endpoint\elastic-endpoint.exe" uninstall --uinstall-token [copied token]

Hopefully that'll do it, unless you really lost the original stack to which the Endpoint (Elastic Defend) was associated.

PS. Could you indicate what was the previous Agent version, and the upgraded Version?

If unsure you can run

"C:\Program Files\Elastic\Endpoint\elastic-endpoint.exe" version
"C:\Program Files\Elastic\Agent\elastic-agent.exe" version

Best,

14

Hi Lesio,

Thank you for your support.

No I have just found this thread when troubleshooting my environment.

The previous agent was 8.11.4 and new one is on 8.13.4.

The thing is there some uninstall tokens that I cannot read in fleet ui, getting below error.

Error when reading Uninstall Token with id ...

Additionally I can fetch all available uninstall tokens using below api.

GET .kibana_ingest/_search?q=type:fleet-uninstall-tokens

But my challenge now is how to decode the token shown by above api call, I tried to decode it using base64 or jwt but cannot.

You can find below an example of token shown by the api call.

"token": "V1E4H/0Z5el3iwSbasHGCWqw5jhmRPv+O9CzknB/O/10bxNT/b87BWOHzU13myulaSDIelLNp8UGtUunFxCBtZsrg00fpXmp/Fs30w3WyfdVWGY/gb4zXEX/DDxQpFMsEJCR6+cB+Dq/5PsN9WDvTBR0nCwrRaDoENZNnx4i"

My approach is to decode all available uninstall tokens then use them one by one in below command to see if it works.

elastic-endpoint.exe" uninstall --uinstall-token <token>

Please support if you know how to decode the token.

Thanks & Best Regards,

Hamid Allaoui

1 Like
15

Let me ask around internally, decoding it should be straightforward

16

Ok and thanks for your support.

17

Unfortunately I've learned that the tokens obtained by the API cannot be decoded on purpose, they are encrypted. This API is just to provide backup & restore option for the tokens.

I'm looking for further steps.

18

Hi Lesio,

Thanks again for your availability to support.

Actually we need this token or any method either to forcely uninstall the endpoint, so that we can reinstall a new agent with required version which is 8.13.4.

Thanks & Best Regards,

Hamid Allaoui

19

Hello Hamid.

We would like to investigate your issue further so I would ask you to change Kibana log level to debug by following these instructions: Examples | Kibana Guide [8.13] | Elastic
By default logs are saved to /var/log/kibana/kibana.log

With debug log set up please visit page in Kibana where uninstall tokens are shown, to capture as much details as possible.

After this is done please lower the log level back to info, because debug generates a lot of information and can quickly take up a lot of disk space.

I will send you the upload link to private message for you to share the log with us.

Regards,
Jure

20

Hi Jure,

Thank you for your support.

For your info we are using ECK for provisioning ELK stacks.

Our setup having issue has 2 kibana pods and I collected their logs using below command and uploaded to the url you shared with me.

kubectl logs kibana-kb-69bb4464d6-xxx > kibana1.logs
kubectl logs kibana-kb-69bb4464d6-yyy > kibana2.logs

Thanks & Best Regards,

Hamid Allaoui