# How to reduce primary storage size

**URL:** <https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 19, 2020, 9:24am UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518 "2020-10-19T09:24:20Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [October 19, 2020, 9:24am UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/1 "2020-10-19T09:24:20Z")

</div>

hi all,  
generally, when I use Logstash I like to remove the message `field` if the `filter` section doesn't contain any error. I do it to reduce the size of every document.  
since this time I will not use Logstash, is there a way to do it? Should I use an ingest pipeline?  
Is there any extra parameter or compression that I can configure?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 19, 2020, 9:38am UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/2 "2020-10-19T09:38:32Z")

</div>

What tool are you using to ingest the data?

---

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [October 19, 2020, 10:02am UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/3 "2020-10-19T10:02:50Z")

</div>

sorry, you are right 🙂  
I'm using Filebeat.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 19, 2020, 10:18am UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/4 "2020-10-19T10:18:50Z")

</div>

I switched your post to #elastic-stack:beats

I'd use this

> **[Drop events | Filebeat Reference \[7.9\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/drop-event.html)**

So the event is not even sent to elasticsearch. Which will also save your network bandwidth. 🙂

---

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [October 19, 2020, 10:32am UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/5 "2020-10-19T10:32:06Z")

</div>

thank you.  
just one doubt: I cannot drop the `message` field using the Filebeat feature, right? I mean, if I would do it I will simply lose the document's content?

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [October 19, 2020, 10:43am UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/6 "2020-10-19T10:43:29Z")

</div>

Yes, if you drop `message` you lose the actual contents of the message. If you want to reduce the size of the event you could drop metadata fields which are not important to you.

---

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [October 20, 2020, 3:34pm UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/7 "2020-10-20T15:34:05Z")

</div>

ok, thanks.  
To modify already existing docs into indices, can I use an ingest pipeline?

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [October 21, 2020, 12:50pm UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/8 "2020-10-21T12:50:56Z")

</div>

Yes, you can reindex your existing docs from indices into another index like this:

```auto
POST _reindex
{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "dest",
    "pipeline": "your_ingest_pipeline"
  }
}

```

Ref: [https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-routing](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-routing)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 18, 2020, 12:51pm UTC](https://discuss.elastic.co/t/how-to-reduce-primary-storage-size/252518/9 "2020-11-18T12:51:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.