Endpoints are enrolling stuck message

Elastic Security Endpoint Security
#1

Hi Elastic team,

I am currently running the latest version of Elastic Agent which is enrolled in the fleet but I don't see any endpoint enrolled yet. The message I get for a few days now in Security/Detections/Administration page is "Endpoints are enrolling. View agents to track progress". But I checked and agent is Online status now.

The steps that I followed up until now are:

  • installing elasticsearch cluster and kibana
  • securing communication between these 2 with a self signed certificate
  • installing elastic agent on a windows 10 machine following the official documentation
  • running the elastic agent in powershell

Can you please help me figure out the root cause of this issue?
If you need more info or screenshots please let me know.

Thanks in advance!
Victor

#2

Hi @VictorG

Did you enable the endpoint security integration in Fleet? From what you wrote it seems you did but I'd like to verify.

Given that you are using a self signed certificate the issue may be that the root certificate is not installed on the host. Is it installed? If not, can you install it to see if that fixes this issue.

When you click on the Agent name in KIbana on the Fleet -> Agents page you will be taken to an activity log for that Agent. Are there any log messages in there related to endpoint-security that show errors?

Is Elastic Endpoint installed and running on the host? If installed it will be in c:\Program Files\Elastic\Endpoint. Logs are written to c:\Program Files\Elastic\Endpoint\state\log\endpoint-*.log. If you have logs flowing let me know and we can dig into them.

#3

I am having the same problem. The root certificate is installed on the host and the intermediate is installed. In the endpoint security I get this in the log
Http.cpp:38 CURL error 60: Error [SSL certificate problem: unable to get local issuer certificate]

#4

Hello @ferullo

I checked the steps you mentioned and in the endpoint-*.log file I am receiving the following:
Elasticsearch connection is down",
HttpLib.cpp:1442 Establishing GET connection to [https://x.x.x.x:9200/_cluster/health]","process":{"pid":11780,"thread":{"id":9352}}}
"message":"Http.cpp:38 CURL error 60: Error [SSL certificate problem: unable to get local issuer certificate]

It seems that is the same error message that was provided by @AirJordan but I want to add something here: the CA certificate was generated with the elasticsearch-certutil command and followed the exact steps from official documentation. Furthermore the communication between elasticsearch and kibana is done properly over https as I can see the metrics logs in kibana (see below the screenshot)

metrics
metrics1474×426 42.1 KB

#5

That is the same for me also. Metrics are collecting fine. I added an IIS integration and most of that works on one of the servers. I understand it is a Beta Product so I don't expect everything to work but I can't get Endpoint Security to really work at all. I have https setup in kibana and elastic and kibana/elastic are talking fine to each other. I did have one client in working status but I stupidly thought the other ones weren't working for another reason and it removed it from the working one also. I will try and dig and see if I can find if I can make the other one work again and see if I can spot the difference.

#6

I ended up getting mine fixed. For me I am doing this in a PoC but I have a root and intermediate cert. For some reason it also wanted the intermediate cert to be import into the root trusted store also, just not the intermediate. Very weird. Sorry I didn't do mine using the cert-util way so I don't know what it generates for a cert.

#7

Hello @VictorG and @AirJordan,

Can you confirm that you have the certificate authority generated by certutil installed on the host?
And that ElasticEndpoint has been restarted since doing so?

I was able to start up an endpoint with the same error message:
Http.cpp:38 CURL error 60: Error [SSL certificate problem: unable to get local issuer certificate]

Then I was able to connect to elasticsearch by doing those two things. First, installing the self-signed certificate authority on the machine running ElasticEndpoint, and second, stopping and starting the ElasticEndpoint service. (Re-reading the system certificate authorities requires a new policy be applied, or ElasticEndpoint restart)

I installed certificate authority by doing the following on my Windows 10 box. Hopefully, the steps are similar for you:

  1. Copy the generated self-signed elasticsearch certificate authority, ca.crt, to the endpoint host.
  2. Double click ca.crt
  3. Click Install Certificate
  4. Select Local Machine
  5. Click Next
  6. Select Place all certificates in the following store
  7. Click Browse
  8. Select Trusted Root Certification Authorities
  9. Click OK
  10. Click Next
  11. Click Finish
  12. See a pop up The import was successful
  13. Click OK to dismiss the pop up
  14. Click OK to dismiss the Certficate window

I stopped and start the ElasticEndpoint service be doing the following:

  1. Start task manager
  2. Select the Services tab
  3. Right Click ElasticEndpoint
  4. Select Stop
  5. Wait for PID to become blank
  6. Right Click ElasticEndpoint
  7. Select Start

I sure hope this helps.

1 Like
Fleet enrollment is done but doesn't appear on Security > Administration > Endpoints
Fleet enrollment is done but doesn't appear on Security > Administration > Endpoints
No alert in security detection dashboards after malware attack
#8

Glad you got it working. And thanks for posting back!

#9

Yes I forgot to add that I had to restart the services but it is working.

#10

Hello @Nick_Berlin,

First of all I want to thank all the people involved in solving this. Second of all I was able to fix this as per your reply. The part that was wrong from my side was that I selected Current user instead of Local machine.

Thanks again for the support!

1 Like
#11

Thanks for diagnosing this! To confirm, before you added the intermediate certificate Agent was able to connect to Fleet and appear active in the Fleet UI, it was just Endpoint Security that failed to send data to Elasticsearch?

#12

Correct. I could see everything working including IIS integrations I added. It was just the Endpoint Security that was stuck. I was seeing it active in Fleet Management. The other issue I do not like is I can have a host with say Winlogbeats and then Elastic-Agent on it and it will show the client twice in the front page of Endpoint security. It will consider say poc.poc.org as one machine and then poc as another yet they are the same machine. Then if I specifically go under hosts and administration I only see it once. It was under the administration where they were not showing up.

#13

Is there a chance that the CA root was installed a user by mistake and installing the intermediate certificate fixed the issue because it was installed for the entire system?

Regarding the Winlogbeat issue, thanks for reporting. I commented on this issue https://github.com/elastic/kibana/issues/77964 highlighting what you saw and have raised it internally.

#14

Is this question aimed at me? I had the root installed at the computer level, installed under trusted root. I had the intermediate installed at computer level installed under intermediate authority. The only way I could get it to work is at the computer level add the intermediate authority to the root folder also, then restart the service. All this was at the computer level not the user level.

#15

Oh yeah, I should have tagged you @AirJordan. Thanks for following up and confirming.

#16

Hello. I am having this same issue; however, I am not getting any error in my logs.

image

image
image1159×127 21.8 KB

I am also using self-signed cert. I have also ensured the cert is installed at the Computer level and not the user level.

#17

@man715 have you by chance changed the namespace from the default value?

#18

I am experiencing the same result. I have 3 agents out on endpoints and they are showing as enrolled but there is no data in the metric-* index and the "Endpoints are enrolling" message persists. (default namespace)

#19

@man715 @SigmazGFX

I'm sorry you're having so much trouble. Would you be able to DM me the Endpoint logs in c:\Program Files\Elastic\Endoint\state\logs? I understand if you're uncomfortable sharing raw logs, I can follow up with log messages for you to look for, but full logs will help resolve this with less back and forth.

#20

That's no problem I can share that with you.
one moment I will send it over