# Offline monitoring or Spooling into a disk
**URL:** https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110
**Category:** Beats
**Tags:** metricbeat
**Created:** [September 11, 2019, 5:03pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110 "2019-09-11T17:03:28Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@RayKishev](https://discuss.elastic.co/u/RayKishev)
#### Post date: [September 11, 2019, 5:03pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/1 "2019-09-11T17:03:28Z")
Hello Elastic team,
I really need your help on this. I am trying to setup the internal Queue, so when my system is offline for some time and after connecting it back to the network, i could still receive a metricbeat data in Kibana, during the system was offline. I configured everything correct, here is the conf
queue.mem:
events: 4096
flush.min\_events: 512
flush.timeout: 5s queue.spool:
file:
path: "${path.data}/spool.dat"
size: 2048MiB
page\_size: 16KiB
write:
buffer\_size: 100MiB
flush.timeout: 5s
flush.events: 1024
but it can only provide data for only 30 minutes. For example if my system disconnected at 10 am, and i connect it back to the network at 12 pm, i can only only see the data up untill 10:30 am.
I ve also tried configure logstash, but samething there as well. Do you have any idea on how we can fix it ?
thank you,
Ratmir
---
### Author:  [@Alex\_Kristiansen](https://discuss.elastic.co/u/Alex_Kristiansen)
#### Post date: [September 11, 2019, 5:28pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/2 "2019-09-11T17:28:01Z")
Your config snippet is a tad difficult to read, but I would make sure have `queue.spool` configured properly and try and set a fairly high file size, which it looks like you have. [https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-internal-queue.html#configuration-internal-queue-spool](https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-internal-queue.html#configuration-internal-queue-spool)
You might also want to make sure you've set `max_retries` properly so you're not dropping events: [https://www.elastic.co/guide/en/beats/metricbeat/7.3/elasticsearch-output.html#\_literal\_max\_retries\_literal](https://www.elastic.co/guide/en/beats/metricbeat/7.3/elasticsearch-output.html#_literal_max_retries_literal)
However, considering your circumstances and the fact that you're disconnected for a number of hours, I would recommend trying to have something between metricbeat and ES that's available for longer periods of time.
---
### Author:  [@RayKishev](https://discuss.elastic.co/u/RayKishev)
#### Post date: [September 11, 2019, 6:13pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/3 "2019-09-11T18:13:00Z")
Thank you very much for replying me.
I have increased the size file to: 4096MiB and also set max\_retries to 500, backoff.max and backoff.init to 120s, but looks like still same output.
What would you recommend to have between metricbeat and ES to help me increase time? I aslo checked the file path where the data while the system is off should go, but coundt find /spool.dat file. I checked the /var/lib/metricbeat and its empty
---
### Author:  [@Alex\_Kristiansen](https://discuss.elastic.co/u/Alex_Kristiansen)
#### Post date: [September 11, 2019, 6:50pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/4 "2019-09-11T18:50:05Z")
> [@RayKishev](#):
>
> What would you recommend to have between metricbeat and ES to help me increase time
Anything that can remain up while ES is down that you can use as a "buffer" for extended periods of time.
Are you sure file spooling is enabled? If you check your metricbeat log, it should print a message warning you that it's on. Also, try commenting out all of the `queue.mem` config and just leaving `queue.spool` configured.
---
### Author:  [@RayKishev](https://discuss.elastic.co/u/RayKishev)
#### Post date: [September 11, 2019, 7:00pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/5 "2019-09-11T19:00:29Z")
didn't see any messages that it's on. Do i have to do anything to enable spooling ? i configured it and restarted metricbeat service, should't it be enabled after that?
Just commented queue.mem, will try to disconnect again
---
### Author:  [@RayKishev](https://discuss.elastic.co/u/RayKishev)
#### Post date: [September 11, 2019, 8:03pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/6 "2019-09-11T20:03:32Z")
still same 25 min. i disconnected it at 12pm and connected back at 1 pm, and it only restores data up until 12:25.
I noticed that even i i will comment all spooling configuration, it still remember those 25 minutes. I think queue.spool configuration not working at all.
---
### Author:  [@Alex\_Kristiansen](https://discuss.elastic.co/u/Alex_Kristiansen)
#### Post date: [September 11, 2019, 9:44pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/7 "2019-09-11T21:44:23Z")
Can you post your entire metricbeat config using the code formatting with backtics?
---
### Author:  [@RayKishev](https://discuss.elastic.co/u/RayKishev)
#### Post date: [September 11, 2019, 10:09pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/8 "2019-09-11T22:09:00Z")
Y mean metricbeat.yml file?
---
### Author:  [@Alex\_Kristiansen](https://discuss.elastic.co/u/Alex_Kristiansen)
#### Post date: [September 11, 2019, 10:30pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/9 "2019-09-11T22:30:03Z")
Yes.
---
### Author:  [@RayKishev](https://discuss.elastic.co/u/RayKishev)
#### Post date: [September 13, 2019, 4:05pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/10 "2019-09-13T16:05:47Z")
Thank you, Alex I figured what was the problem. Apparently it was something with permissions.
The only question i have now is, the size of my /spool.dat file increased by 300MB and after connecting it back to the network, it didn't empty /spool.dat file. How can we set events to immediately remove from the queue?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 11, 2019, 4:05pm UTC](https://discuss.elastic.co/t/offline-monitoring-or-spooling-into-a-disk/199110/11 "2019-10-11T16:05:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.