No alert in security detection dashboards after malware attack

[Diana_Dragoiu]

Hello ES team,
After installing and enrolling Elastic Agent 7.10 we launched an Mimikatz.exe on our Win10 machine and received an ES Malware Alert but we do not receive an alert in Security Detections tab in Kibana. The rule: Malware - Detected - Endpoint Security was activated and can't be edited.

In Security->Administration tab there are no Endpoints enrolled.

In Data streams tab the elastic_agent.endpoint-security is not integrated to endpoint.

We tried to recreate the case(link) as bellow but no alert is shown -> https://newtonpaul.com/how-to-install-elastic-siem-and-elastic-edr/#Installing_Elastic_EDR_SIEM

We enrolled the agent using a CA self signed certificate using:
.\elastic-agent.exe enroll https://x.x.x.x:5601 a2kzelJuWUJsSUFCcjI3UHBaYVc6ekdkUW1YMVFUVldHQnNnWXBMLW9wZw== -a 'C:\Program Files\Elastic\Agent\ca.crt'

The .yml files are below:
elastic-agent.yml

elastic-endpoint.yml

fleet.yml

Could you be so kind and help me?

Thank you!

[ferullo (Daniel Ferullo)]

Hi @Diana_Dragoiu

It seems like your issue might be that Elastic Endpoint is not able to send data to Elasticsearch from the information you provide (thanks for it all!). It appears Endpoint is working it just can't tell Kibana it is working.

Could you look in the Endpoint logs to see what might be happening? On Windows Endpoint logs are placed in c:\Program Files\Elastic\Endpoint\state\log\endpoint-*.log

A few log messages to search for are No valid comms client available, Dropping rejected document, Elasticsearch connection is down, and Sent XXX documents to Elasticsearch (where XXX is a number that changes per log message, I'd expect it to be 0 in all cases for you but if it's not that would be interesting). When those logs appear the messages before and after would also be interesting to know. In particular just before some of those messages appear you will probably see messages related to the HTTP session in use. That should help us narrow down why Endpoint is not putting data into Elasticsearch.

[Diana_Dragoiu]

Hello @ferullo,
Thank you for your reply.

From last endpoint-*.log I get the following line(ordered):

Elasticsearch connection is down:

ssl.params not found:

ssl problem. Unable to get the ssl issuer:

Agent issues:
One day before:

At this moment from Services:

My thoughts are that because of the ca.crt self signed certificate is this problem.

From the link( https://newtonpaul.com/how-to-install-elastic-siem-and-elastic-edr/#Installing_Elastic_EDR_SIEM) I have done:

Imported ca.crt here -> Trusted Root Certification Authorities > Certificates
Set the local policy : Security Settings > Public Key Policies > Certificate Path Validation Settings.
Enrolled the agent with the following option : -a 'C:\Program Files\Elastic\Agent\ca.crt'

elastic-endpoint.yaml complete:

Your help is appreciated. And we are planning to use the Agent in the future in Production. Any estimated date when the agent will move from beta state?

[warkolm (Mark Walkom)]

Please don't post pictures of text, they are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them

[Diana_Dragoiu]

Hello @warkolm,

Sorry about that. Did not know.
So I'll post some text from endpoint*.log:
tp.cpp"}}},"message":"Http.cpp:38 CURL error 60: Error [SSL certificate problem: unable to get local issuer certificate]","process":{"pid":10012,"thread":{"id":3828}}}
{"@timestamp":"2020-12-11T00:44:18.5332287Z","agent":{"id":"5b05aef9-fc59-0d88-f268-461975e80fae","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"notice","origin":{"file":{"line":84,"name":"BulkQueueConsumer.cpp"}}},"message":"BulkQueueConsumer.cpp:84 Elasticsearch connection is down","process":{"pid":10012,"thread":{"id":3828}}}

CURL error 60: Error [SSL certificate problem: unable to get local issuer certificate]","process":{"pid":592,"thread":{"id":1384}}}
{"@timestamp":"2020-12-09T14:54:55.0468446Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"notice","origin":{"file":{"line":84,"name":"BulkQueueConsumer.cpp"}}},"message":"BulkQueueConsumer.cpp:84 Elasticsearch connection is down","process":{"pid":592,"thread":{"id":1384}}}
{"@timestamp":"2020-12-09T14:54:55.1093331Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"info","origin":{"file":{"line":83,"name":"AgentConnectionInfo.cpp"}}},"message":"AgentConnectionInfo.cpp:83 Failed to find connection to validate. Is Agent listening on 127.0.0.1:6788?","process":{"pid":592,"thread":{"id":2080}}}
{"@timestamp":"2020-12-09T14:54:55.1093331Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"error","origin":{"file":{"line":107,"name":"AgentConnectionInfo.cpp"}}},"message":"AgentConnectionInfo.cpp:107 Agent process is not root/admin or validation failed, disconnecting","process":{"pid":592,"thread":{"id":2080}}}
{"@timestamp":"2020-12-09T14:54:55.1093331Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"warning","origin":{"file":{"line":164,"name":"AgentConnectionInfo.cpp"}}},"message":"AgentConnectionInfo.cpp:164 Failed to established stage 1 connection to agent","process":{"pid":592,"thread":{"id":2080}}}
{"@timestamp":"2020-12-09T14:54:55.1093331Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"error","origin":{"file":{"line":538,"name":"AgentComms.cpp"}}},"message":"AgentComms.cpp:538 Unable to retrieve connection info from Agent(Agent is not running as root)","process":{"pid":592,"thread":{"id":2080}}}
{"@timestamp":"2020-12-09T14:55:00.0468855Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"info","origin":{"file":{"line":1442,"name":"HttpLib.cpp"}}},"message":"HttpLib.cpp:1442 Establishing GET connection to [https://192.168.224.10:9200/_cluster/health]","process":{"pid":592,"thread":{"id":1384}}}
{"@timestamp":"2020-12-09T14:55:00.0468855Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"error","origin":{"file":{"line":38,"name":"Http.cpp"}}},"message":"Http.cpp:38 CURL error 60: Error [SSL certificate problem: unable to get local issuer certificate]","process":{"pid":592,"thread":{"id":1384}}}
{"@timestamp":"2020-12-09T14:55:00.0468855Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"notice","origin":{"file":{"line":84,"name":"BulkQueueConsumer.cpp"}}},"message":"BulkQueueConsumer.cpp:84 Elasticsearch connection is down","process":{"pid":592,"thread":{"id":1384}}}
{"@timestamp":"2020-12-09T14:55:01.1094495Z","agent":{"id":"93924273-5e90-c21c-3748-ec0f2c92b9e8","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"info","origin":{"file":{"line":83,"name":"AgentConnectionInfo.cpp"}}},"message":"AgentConnectionInfo.cpp:83 Failed to find connection to validate. Is Agent listening on 127.0.0.1:6788?"

Regards,
Diana

[Nick_Berlin]

Hello @Diana_Dragoiu,

I'm not sure if C:\Program Files\Elastic\Agent\ca.crt is the same certificate authority that is generated by the elasticsearch instance. Can you confirm that?

The ElasticEndpoint makes a direct connection to elasticsearch and according to the logs above it's failing to do so because it cannot find the proper certificate authority in the host's trusted root certificates.

Please ensure the certificate authority from elasticsearch is installed and that ElasticEndpoint service be stopped and started after doing so.

Here's a link to steps I used to get a Windows 10 ElasticEndpoint connected to elasticsearch.

Sure hope this helps.

[Diana_Dragoiu]

Hello @Nick_Berlin,

Finally it works. I have installed the ca.crt before as User not Local Machine and I added the ca.crt certificate that already worked for Kibana.

Many thanks to you and your team !

[Nick_Berlin]

Excellent, glad it's working. Thanks for the feedback about User vs Local Machine.