# Efficient storage

**URL:** <https://discuss.elastic.co/t/efficient-storage/204684>\
**Category:** Elasticsearch\
**Created:** [October 22, 2019, 3:31pm UTC](https://discuss.elastic.co/t/efficient-storage/204684 "2019-10-22T15:31:39Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![eddie4](https://avatars.discourse-cdn.com/v4/letter/e/838e76/32.png) [@eddie4](https://discuss.elastic.co/u/eddie4)\
**Post date:** [October 22, 2019, 3:31pm UTC](https://discuss.elastic.co/t/efficient-storage/204684/1 "2019-10-22T15:31:39Z")

</div>

Hello, am planning on storing 1,5TB of textfiles in Elasticsearch. However I do not have a lot of space to spare above the text file size. Now I have been reading up on elastic search and how to minimize storage and I have already made the following changes

Removed unnecessary fields  
Removed \_source  
Enabled best\_compression

This has decreased my the diskspace used by elasticsearch from 7 times the size of the text files down to 3 times the text file. But I was hoping there is a way to compress this even further.

The path is the location where the original textfile is located and is required. And the messageline is currently filled with close to random data very little text that is reused(But must be word searchable). I have thought about splitting the message field but I run into grok running after mutate. Which means if i mutate remove message that grok can't parse the fields. (But that is another problem)

```
{
  "mapping": {
    "properties": {
      "message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "path": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

```

Any suggestions are welcome.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 23, 2019, 1:22am UTC](https://discuss.elastic.co/t/efficient-storage/204684/2 "2019-10-23T01:22:43Z")

</div>

Do you need the keyword subfields? If not, I'd remove them.

---

<div class="post-metadata">

**Author:** ![eddie4](https://avatars.discourse-cdn.com/v4/letter/e/838e76/32.png) [@eddie4](https://discuss.elastic.co/u/eddie4)\
**Post date:** [October 24, 2019, 7:43am UTC](https://discuss.elastic.co/t/efficient-storage/204684/3 "2019-10-24T07:43:33Z")

</div>

As i understand it they are required to search the field arent they? if not how would one go about removing them? I think I tried at one point but wasn't allowed to.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 24, 2019, 8:13am UTC](https://discuss.elastic.co/t/efficient-storage/204684/4 "2019-10-24T08:13:49Z")

</div>

You need keyword fields if you need to aggregate or sort.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 21, 2019, 8:13am UTC](https://discuss.elastic.co/t/efficient-storage/204684/5 "2019-11-21T08:13:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.