# Elastic Agent filling up disk space with logs, disaster
**URL:** https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668
**Category:** Endpoint Security
**Created:** [June 22, 2021, 2:33pm UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668 "2021-06-22T14:33:19Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Achilleas](https://discuss.elastic.co/u/Achilleas)
#### Post date: [June 22, 2021, 2:33pm UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668/1 "2021-06-22T14:33:19Z")
We had a meltdown on one of our servers, where the elastic agent filled up the disk space with logs (36GB of text logfiles, in 2 months), from a trial we had on the Elastic Cloud.
I know I am to blame for installing an experimental beta on a production server. Anyone knows how this happened? (Trial ended, so elasticsearch cluster went offline, possible cause?)
Anyone knows how to configure on how to prevent this from happening again?
---
### Author:  [@ferullo](https://discuss.elastic.co/u/ferullo)
#### Post date: [June 22, 2021, 2:59pm UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668/2 "2021-06-22T14:59:10Z")
Hi @Achilleas Thanks for trying out Agent. I'm sorry to hear that happened to you. Could you please share the path(s) to the files that filled up your disk to help us narrow down where the problem is?
---
### Author:  [@rugenl](https://discuss.elastic.co/u/rugenl)
#### Post date: [June 22, 2021, 3:11pm UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668/3 "2021-06-22T15:11:30Z")
Agents will probably log a lot of messages if they can't send events, so when the cluster went offline, the messages started.
I've noticed that Elastic doesn't send logrotate configs for their logs. If you're LInux, I'd suggest you always define logrotate policy for anything that produces log files.
If you're Windows, I'm sorry 🙂 ( I don't know the alternative for logrotate there....)
---
### Author:  [@Achilleas](https://discuss.elastic.co/u/Achilleas)
#### Post date: [June 22, 2021, 3:39pm UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668/4 "2021-06-22T15:39:35Z")
I am afraid I deleted them ASAP as I really had to do damage control.
I do remember they were under "C:\Program Files\Elastic....\logs"
I guess there should be a limit to log retention in case of ELK unreachable...
---
### Author:  [@ferullo](https://discuss.elastic.co/u/ferullo)
#### Post date: [June 23, 2021, 3:19pm UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668/5 "2021-06-23T15:19:25Z")
Thank you for that path, even though it was only from memory and a bit incomplete. I was not able to reproduce that path filling up the disk under normal circumstances. However, when I mimicked a possible behavior some other antivirus or backup software on your system may have I was able to get that path to fill up.
We'll put in place mitigations so that if it this was caused by another application on your computer that that it won't lead to your hard drive filling up in the future.
---
### Author:  [@Achilleas](https://discuss.elastic.co/u/Achilleas)
#### Post date: [June 24, 2021, 8:30am UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668/6 "2021-06-24T08:30:01Z")
Thank you for looking into this. No other security software was present at the server. To my eyes, it is obvious that these accumulating logs were the logs that could not be sent to elasticsearch. So is there a limit implemented there by default, or are they stacking up indefinately?
---
### Author:  [@ferullo](https://discuss.elastic.co/u/ferullo)
#### Post date: [June 28, 2021, 5:03pm UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668/7 "2021-06-28T17:03:52Z")
I missed before that there are two `logs` directories under `c:\Program Files\Elastic`. The one I was referring to is under `c:\Program Files\Elastic\Endpoint`. There is another under `c:\Program Files\Elastic\Agent`.
Both have max size caps for the files under the directory. The Endpoint one should never grow above 125MB (a 100MB limit , with log files up to 25MB files in size leading to possibly 125MB of data at times). The Agent one 70MB (up to 7 10MB files, configurable with `logging.files.keepfiles` option).
So, this is a long winded way of saying that whatever you saw was a bug. If you see it reproduce again please reach out and let us know. Key to us understanding where the bug is will be knowing a full directory listing (including file sizes) for everything under `c:\Program Files\Elastic`.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 26, 2021, 5:03pm UTC](https://discuss.elastic.co/t/elastic-agent-filling-up-disk-space-with-logs-disaster/276668/8 "2021-07-26T17:03:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.