Failed. Kibana 7.13.1 migration from 7.13.0 failed. Testing setup. Single node. Fleet policies with Elastic-Endpoint only. Agent log's set to enable on settings to upload. When the Elastic instance went offline for updates all agents started to buffer again in memory "Filebeat". This forced all ma…

7.13 offline agent issues. Not caused by Endpoint but by Agent. Filebeat duplicate process is back. This is causing the read/write to continue endlessly as one writes updates and the other reads. Also makes me wonder why Elastic-Endpoint run's as the user and not as system? You log out and the mac…

[image] PublicName: Also makes me wonder why Elastic-Endpoint run's as the user and not as system? You log out and the machine is no longer protected which is common to have server's never logged in. This just showed up in 7.13. elastic-endpoint.exe is launched via a service as the SYSTEM use…

Can you share details about running processes, CPU/memory usage before, during, and after the upgrade? Are these physical test machines or VMs? How big are those?

Nope I do mean as the user. It's not all the time I'll PM you a screen shot on the next one I see. Due to usernames it won't be public. It's one that makes you scratch your head. It's interesting to note that at you will end up with 2 instances of Endpoint running. This does not end well for obvious…

Server 2016 with IIS or File services installed are generally when this happens. We do not nest machine services so no shared process/services or conflicts. 2vCPU/8GB Ram/44Gb HD is the base we use if it's windows only services. 7.12.1 was on average on the dev servers was sitting at 1% CPU Endpoin…

[image] PublicName: Nope I do mean as the user. It's not all the time I'll PM you a screen shot on the next one I see. Due to usernames it won't be public. It's one that makes you scratch your head. It's interesting to note that at you will end up with 2 instances of Endpoint running. This does …

I've had both push the machine into memory starvation mode. Filebeat tends to be by far the most common but endpoint will start in if you leave the machine long enough. At which point your users have caught you long before that point. I left a few devices over the weekend and came in to see endpoint…

seems like you're hitting multiple issues here. can you check if you see 2 or 3 filebeat/metricbeat processes running. if so this was fixed recently. aside from that we can see what you describe with cumulating memory. this may be related to issue above or Investigate cummulating memory when proce…

I'll be honest I stopped using the agent and went back to legacy beats even in dev. The amount of problems it causes is far to much to keep spending time on. Normal day job is hitting the frantic season so testing beta is out the window for awhile. The disk space issue where it killed the drive on …

I just hit the disk filling issue too - got about 2x10^6 events and my server sending a crazy amount of logs to ES

Endpoint 7.13 migration to 7.13.1 Lesson learned with Fleet “On-Prim” -Bad

Elastic Security

ferullo (Daniel Ferullo) June 23, 2021, 3:23pm 16

I tried both. When looking into this other, similar issue (Elastic Agent filling up disk space with logs, disaster) I stumbled upon an issue that could have been what caused the disk size issue you saw, we'll put in a mitigation for that.

Topic		Replies	Views
Endpoint 7.12.x migration to 7.13 Lesson learned with Fleet "On-Prim" Elastic Security fleet	7	838	June 11, 2021
Elastic Agent 7.14 -- Strange bug during enrollment "Elastic fleet agent bug" Elastic Security docker , fleet	15	3165	September 7, 2021
Fleet Agents hitting 100% CPU usage Beats fleet , elastic-agent	19	5043	May 4, 2021
Elastic Agent - critical issues, filling up hard drive space Endpoint Security	1	1065	January 3, 2022
Elastic-agent.exe not running on target Elastic Security fleet	14	2097	August 3, 2021

Endpoint 7.13 migration to 7.13.1 Lesson learned with Fleet “On-Prim” -Bad

Related topics