# How to store metricbeat data in local disk

**URL:** <https://discuss.elastic.co/t/how-to-store-metricbeat-data-in-local-disk/142682>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [August 2, 2018, 6:14am UTC](https://discuss.elastic.co/t/how-to-store-metricbeat-data-in-local-disk/142682 "2018-08-02T06:14:04Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\
**Post date:** [August 2, 2018, 6:14am UTC](https://discuss.elastic.co/t/how-to-store-metricbeat-data-in-local-disk/142682/1 "2018-08-02T06:14:05Z")

</div>

Hi All,

We had a discussion in meeting about the metricbeat data sending through elasticsearch output and if the output elasticsearch is went down where the metric data will store?

I was reading this article,

[https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-internal-queue.html](https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-internal-queue.html)

In that by default the metricbeat uses internal queue to store events in memory before publishing them. For our case we will be capturing metrics data and sending to elasticsearch for the period interval of every 5 mins.

What is the idle number I should defined in the events memory queue?  
The below sample Configuration can store upto 4096 events if I want to store all the events what should I need to do?

Please correct me If my understanding is wrong.

```
queue.mem:
  events: 4096
  flush.min_events: 512
  flush.timeout: 5s 

```

What will happen if I don't specify the no of events in the queue?

Will it store all the metrics data untill the output of elasticsearch comes up?

Also there is an another option in metricbeat called [file spool queue](https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-internal-queue.html#configuration-internal-queue-spool) where we can store all the events on the disk

The below sample configuration can store the events upto 512MB. Can I specifiy the size of 1GB ??  
so that the events will be stored in the local and once the output of elasticsearch comes up all the data will be flushed to the elasticsearch based on the write buffer

```
queue.spool:
  file:
    path: "${path.data}/spool.dat"
    size: 512MiB
    page_size: 16KiB
  write:
    buffer_size: 10MiB
    flush.timeout: 5s
    flush.events: 1024

```

Please let me know your thoughts it would be helpful.

Regards,  
Ganeshbabu R

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [August 2, 2018, 8:42am UTC](https://discuss.elastic.co/t/how-to-store-metricbeat-data-in-local-disk/142682/2 "2018-08-02T08:42:38Z")

</div>

Yes, you can define the size of the spool file. Just take into account that once the spool file is full, Metricbeat will start dropping new events.

You can do the math of what your retention period will be for a certain size, based on the traffic amount specific to your settings.

Best regards

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 2, 2018, 8:57am UTC](https://discuss.elastic.co/t/how-to-store-metricbeat-data-in-local-disk/142682/3 "2018-08-02T08:57:38Z")

</div>

Does the file spools queue works?

I have same concern, [Metricbeat fail safe/over function on output](https://discuss.elastic.co/t/metricbeat-fail-safe-over-function-on-output/142466/8).

But got an error during implementation

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [August 2, 2018, 9:54am UTC](https://discuss.elastic.co/t/how-to-store-metricbeat-data-in-local-disk/142682/4 "2018-08-02T09:54:26Z")

</div>

Can you please share your settings and the error?

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 2, 2018, 9:56am UTC](https://discuss.elastic.co/t/how-to-store-metricbeat-data-in-local-disk/142682/5 "2018-08-02T09:56:40Z")

</div>

It is on the thread.

kindly check this link

> [Metricbeat fail safe/over function on output - #8 by jogoinar10](https://discuss.elastic.co/t/metricbeat-fail-safe-over-function-on-output/142466/8)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 30, 2018, 9:56am UTC](https://discuss.elastic.co/t/how-to-store-metricbeat-data-in-local-disk/142682/6 "2018-08-30T09:56:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.