# Why Binary data with doc value cost so lots of disk space? Or could I use other type instead? **URL:** \ **Category:** Elasticsearch\ **Created:** [June 11, 2020, 4:03am UTC](https://discuss.elastic.co/t/why-binary-data-with-doc-value-cost-so-lots-of-disk-space-or-could-i-use-other-type-instead/236612 "2020-06-11T04:03:00Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![nooneuse](https://avatars.discourse-cdn.com/v4/letter/n/a5b964/32.png) [@nooneuse](https://discuss.elastic.co/u/nooneuse)\ **Post date:** [June 11, 2020, 4:03am UTC](https://discuss.elastic.co/t/why-binary-data-with-doc-value-cost-so-lots-of-disk-space-or-could-i-use-other-type-instead/236612/1 "2020-06-11T04:03:00Z")
Hello, (My elasticsearch version is 6.8.0) I have a lots of DataSketches binary data stored in an index as binary field. I need to deserialize them and combine them together to get the total estimate of them. So I try to wrote a DataSketches aggregation plugin for elasticsearch to combine and calculate them, the result is good, but i found a strange thing about binary data field. I have to open the doc\_value for my binary field, because i need to aggregate those field. But it seems cost too much disk space, for example, 10GB without [`_source`]. I put the same data into druid, an open-source OLAP engine which can also do this job, and discovered those data just cost it only 50MB to store without source data. And, If i close the doc value of this field, it cost 200MB in elasticsearch, and that is acceptable. I don't understand why doc value need so many space, and I don't want to give up(use elasticsearch to process DataSketches objects), cause i find out the elasticsearch with my plugin has a better performance than druid. I realized, the doc value of es binary data type is stored in lucene as sorted-binary doc value(see: [Lucene70DocValuesFormat (Lucene 7.7.0 API)](https://lucene.apache.org/core/7_7_0/core/org/apache/lucene/codecs/lucene70/Lucene70DocValuesFormat.html)), for sorting, aggregating and scipting. > And lucene's sorted doc value is"a mapping of ordinals to deduplicated terms is written as Prefix-compressed Binary, along with the per-document ordinals written using one of the numeric strategies above". the ordinals of My datasketch objects would be very large, cause they can't duplicate, they are really difference. What could I do to reduce the disk space cost of binary data type? Or could I use other data type or way to make the things work without cost so many disk space in elasticsearch? Please help me and give me some advice, thank you very much. If y ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [June 11, 2020, 3:13pm UTC](https://discuss.elastic.co/t/why-binary-data-with-doc-value-cost-so-lots-of-disk-space-or-could-i-use-other-type-instead/236612/2 "2020-06-11T15:13:52Z")
Compression was [recently added](https://issues.apache.org/jira/browse/LUCENE-9211) to binary doc values in Lucene but is currently being [revisited](https://github.com/apache/lucene-solr/pull/1543) as not all use cases want to trade speed for disk space savings. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 9, 2020, 3:13pm UTC](https://discuss.elastic.co/t/why-binary-data-with-doc-value-cost-so-lots-of-disk-space-or-could-i-use-other-type-instead/236612/3 "2020-07-09T15:13:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.