# Stored logs and compression

**URL:** <https://discuss.elastic.co/t/stored-logs-and-compression/101294>\
**Category:** Elasticsearch\
**Created:** [September 21, 2017, 9:00am UTC](https://discuss.elastic.co/t/stored-logs-and-compression/101294 "2017-09-21T09:00:57Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sbampa](https://avatars.discourse-cdn.com/v4/letter/s/8e7dd6/32.png) [@sbampa](https://discuss.elastic.co/u/sbampa)\
**Post date:** [September 21, 2017, 9:00am UTC](https://discuss.elastic.co/t/stored-logs-and-compression/101294/1 "2017-09-21T09:00:57Z")

</div>

Hi all,  
can someone help me with compression data in ES?  
I'm storing around 100 GB per day, with a retention of 90 days.

I need to compress the data.  
Already set index.codec: best\_compression  
Some tips ??

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 21, 2017, 9:06am UTC](https://discuss.elastic.co/t/stored-logs-and-compression/101294/2 "2017-09-21T09:06:53Z")

</div>

The size data takes up on disk depends on the data as well as the mappings used. Have a look at [this blog post about how enrichment and mappings affect storage size](https://www.elastic.co/blog/filebeat-modiles-access-logs-and-elasticsearch-storage-requirements).

---

<div class="post-metadata">

**Author:** ![sbampa](https://avatars.discourse-cdn.com/v4/letter/s/8e7dd6/32.png) [@sbampa](https://discuss.elastic.co/u/sbampa)\
**Post date:** [September 21, 2017, 9:30am UTC](https://discuss.elastic.co/t/stored-logs-and-compression/101294/3 "2017-09-21T09:30:26Z")

</div>

Hi,  
this is the json of an entry:

{  
"\_index": "pippo-2017.09.21",  
"\_type": "log",  
"\_id": "AV6jrCxJrdYxfnal\_XGx",  
"\_version": 1,  
"\_score": null,  
"_source": {  
"@timestamp": "2017-09-21T09:01:07.103Z",  
"offset": 5181412256,  
"level": "DEBUG",  
"@version": "1",  
"beat": {  
"name": "filebeats-test.farm",  
"hostname": "filebeats-test.farm",  
"version": "5.6.1"  
},  
"input\_type": "log",  
"host": "filebeats-test.farm",  
"source": "/opt/logs/jboss/server/default/logs/server.log",  
"message": "2017-09-21 11:01:07,103 DEBUG [adapters.pspv.PspvHazelcastAdapter] (MULTICAST\_SRV_(261-0-PSPV)) FixedOddVO cache updated",  
"type": "log",  
"tags": [  
"beats\_input\_codec\_plain\_applied"  
]  
},  
"fields": {  
"@timestamp": [  
1505984467103  
]  
},  
"sort": [  
1505984467103  
]  
}

For my purpose i need to have "\_all" enabled, the timestamp, source and message.  
How can i delete all other fields?  
I read a lot of doc, but i cannot find a procedure....☹

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 19, 2017, 9:31am UTC](https://discuss.elastic.co/t/stored-logs-and-compression/101294/4 "2017-10-19T09:31:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.