Hi ! I'm kind of new to the whole elastic stack and I was tasked to put in place some sort of log monitoring and siem solution.
After deploying the stack on some test hosts, everything was fine so I deployed the agent on my whole infrastructure. Then when I went to try the elastic-defend integration, that's where everything went array.
To help you better understand my situation, everything is on premise on multiple virtual machines. I have a cluster of 2 nodes of elasticsearch, one kibana machine that is also the fleet-server.
On that fleet, there are 4 agent policies with their integrations:
- Fleet-server policy [Fleet server; System]
- Windows policy [Windows; Network Packet Capture; System]
- Linux policy [Auditd Manager; Network Packet Capture; System]
- Test policy [Elastic Defend; Auditd Manager; Network Packet Capture; System]
As you can see, only the Test Policy has Elastic Defend, but for some reason I don't understand, the service elastic-endpoint has been installed on all my hosts outside of that policy. And now it's having exclusion issues with our EDR.
We're testing the exclusion list as I'm posting this, but I really want to understand why the elastic Endpoint installed itself on all our hosts.
Hi @Pinpin ,
Thanks for all that background context. This shouldn't be happening and isn't something we've seen any other reports of; Elastic Endpoint should only be installed on hosts with Elastic Defend in their Agent policy.
I have a few questions.
- In Fleet, can you double check a host with Elastic Endpoint that shouldn't have it. (In other words can you go to Fleet -> Agents, select the Agent policy for the host in the table and verify you don't see Elastic Defend, Elastic Endpoint Security, or Elastic Endpoint and Cloud Security in the list).
- Did these hosts ever have Elastic Defend added to their Agent policy?
- For one of the affected hosts can you collect Agent Diagnostics (go to Fleet -> Agents -> Select the host -> Diagnostics tab -> Request Diagnostics .zip). In the diagnostics you'll see the Agent configuration. Does it include information about endpoint in it?
- What version of the Elastic Stack are you running?
- Do you have any reproduction steps?
You can DM me any files / info you are concerned with posting publicly.
Honestly the situation occured 2 weeks ago, we had to uninstall all elastic-agent and endpoint because our hosts were barely usable at that point.
I can only answer your questions from memory since we wiped all our elastic-agent and did a fresh deployment.
- None of integration you mentioned were installed at the time of the issue.
- I did have Elastic Defend added to their policy at some point but I removed it days prior to the incident. Some hosts weren't even enrolled when I added and removed the Elastic Defend.
- None to share.
- version 8.12.0
- I did some tests on a smaller scale, but couldn't reproduce the incident.
So I came to wonder, could this have been an issues with the policy changes that I made ? Maybe I didn't allocate enough resources for my cluster and some changes in the policy weren't taken into account or overlaped?
Also, I remember that it occured on the same day that I updated a bunch of elastic-agent.
If you try Elastic Defend/Endpoint again (please do!) definitely reach out if you see this again. Like I said, it's not something that should happen, there is no reason it would, and we've never heard any one else report this. Since you installed Elastic Defend to those policies previously and you had host enrolled/unenrolled during the time you might have seen some residual state transitions still being processed.
You've mentioned having another EDR product. It's likely the problem you experience were Endpoint and your other EDR interacting with each other. It's common for antivirus software to need to be configured to ignore each other so they don't monitor each other and create feedback loops. A more lengthy description of the issue and what to do is available here. The document is Windows focused by the principles are the same for Linux and macOS.
I'm currently trying Elastic Defend again, on a few test machines. Will definitely post again if the problem occurs. Thanks for your time and feed back !