# A question on the spooling to disk **URL:** \ **Category:** Beats\ **Tags:** beats-development\ **Created:** [September 18, 2018, 8:02pm UTC](https://discuss.elastic.co/t/a-question-on-the-spooling-to-disk/149018 "2018-09-18T20:02:24Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\ **Post date:** [September 18, 2018, 8:02pm UTC](https://discuss.elastic.co/t/a-question-on-the-spooling-to-disk/149018/1 "2018-09-18T20:02:24Z")
When events cannot be send to Elastic Search, events are nicely written to disk. On restart the events are send to ElasticSearch. I was wondering, do we always need to restart? Or can we set an option to retry to submit after XX time period? Or when other events are again successfully submitted, can we resubmit the pending events. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [September 19, 2018, 10:05am UTC](https://discuss.elastic.co/t/a-question-on-the-spooling-to-disk/149018/2 "2018-09-19T10:05:58Z")
Hi @sentient, Do you mean restarting filebeat? There is no need to restart it. Filebeat keeps retrying to connect till Elasticsearch is available again, when it is, and if queue spool is being used, pending events are submitted. This reconnection is not inmediate, but it should be a matter of seconds to happen. You can see reconnection attempts in the logs. Are you observing a different behaviour? ---
**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\ **Post date:** [September 19, 2018, 2:35pm UTC](https://discuss.elastic.co/t/a-question-on-the-spooling-to-disk/149018/3 "2018-09-19T14:35:18Z")
Jaime, Thanks for your reply. I am actually developing a custom beat to send Statsd data into Elastic search ([https://github.com/sentient/statsdbeat](https://github.com/sentient/statsdbeat)). Not complete yet, just under development.... Initially I was writing my own disk storage but then I found out about the spooling functionality (just what I was looking for). So if you write your own beat and use the spooling queue, how do I control the retry functionality. I did notice in the logs things about a 'retryer' sending signals. Do I have to act on this (or is this something completely different) ```auto 2018-09-18T16:48:29.784-0700 INFO [publish] pipeline/retry.go:172 retryer: send unwait-signal to consumer 2018-09-18T16:48:29.784-0700 INFO [publish] pipeline/retry.go:174 done 2018-09-18T16:48:29.784-0700 INFO [publish] pipeline/retry.go:149 retryer: send wait signal to consumer 2018-09-18T16:48:29.784-0700 INFO [publish] pipeline/retry.go:151 done ``` ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [September 19, 2018, 4:47pm UTC](https://discuss.elastic.co/t/a-question-on-the-spooling-to-disk/149018/4 "2018-09-19T16:47:15Z")
Oh, this is nice, I was assuming that you were trying it with filebeat, not your own beat 🙂 To use the disk queue it should be enough with adding `queue.spool: ~` to your configuration. But retry logic is part of the output and this is controlled there, not in the queue. You can find for example some parameters for that in the [Elasticsearch output](https://www.elastic.co/guide/en/beats/filebeat/6.4/elasticsearch-output.html), like `max_retries`, `backoff.init` or `backoff.max`. By the way, have you considered to add a statsd metricset to metricbeat instead of implementing a whole beat? There is for example a [module for graphite](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-graphite.html). ---
**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\ **Post date:** [September 19, 2018, 6:09pm UTC](https://discuss.elastic.co/t/a-question-on-the-spooling-to-disk/149018/5 "2018-09-19T18:09:26Z")
Thanks again Jaime, Great tip for the metricbeat module. I will a look into that. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 17, 2018, 6:09pm UTC](https://discuss.elastic.co/t/a-question-on-the-spooling-to-disk/149018/6 "2018-10-17T18:09:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.