# Data Stream data retrieval and compression **URL:** \ **Category:** Elasticsearch\ **Tags:** ilm-index-lifecycle-management, datastreams\ **Created:** [June 17, 2021, 1:44pm UTC](https://discuss.elastic.co/t/data-stream-data-retrieval-and-compression/276272 "2021-06-17T13:44:28Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![prithvirajsivarajan](https://avatars.discourse-cdn.com/v4/letter/p/8e8cbc/32.png) [@prithvirajsivarajan](https://discuss.elastic.co/u/prithvirajsivarajan)\ **Post date:** [June 17, 2021, 1:44pm UTC](https://discuss.elastic.co/t/data-stream-data-retrieval-and-compression/276272/1 "2021-06-17T13:44:28Z")
I'm working on a data stream. I have created a test ILM policy. Have a couple of question regarding that. 1. Is there a huge difference time lag while retrieving data from hot, warm, cold phase indices. I tried inserting a few documents and used shrink API to reduce number of shards to 1 in the warm phase. But I'm not experiencing much time difference in retrieving the data from warm, hot and cold phases. Is that the case or will it differ when there is large amount of data. How much of a time difference can we expect for data retrieval between the phases. 2. I'm trying to see if there is a compression technology on data stream. ie, the data on the cold phase is not needed for searching anymore. Can we zip that data so that we can get more disk space on the cluster and can store store more data to the cold phase. Does the size reduces when we move index to cold phase itself? Or the shrink API is actually used for reducing the size of the index(I'm not sure if reducing the number of shards reduces the size of the index) ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 18, 2021, 2:45am UTC](https://discuss.elastic.co/t/data-stream-data-retrieval-and-compression/276272/2 "2021-06-18T02:45:23Z")
> [@prithvirajsivarajan](#): > > Is there a huge difference time lag while retrieving data from hot, warm, cold phase indices Retrieving how exactly? Are the phases on the same hosts, or do you have different hardware profiles for each phase? > [@prithvirajsivarajan](#): > > I'm trying to see if there is a compression technology on data stream Elasticsearch compresses by default. You cannot zip the underlying data without losing access to it. > [@prithvirajsivarajan](#): > > Does the size reduces when we move index to cold phase itself? Or the shrink API is actually used for reducing the size of the index The `_shrink` API is used to reduce the number of shards, it should also help to minimise the size of the index. ---
**Author:** ![prithvirajsivarajan](https://avatars.discourse-cdn.com/v4/letter/p/8e8cbc/32.png) [@prithvirajsivarajan](https://discuss.elastic.co/u/prithvirajsivarajan)\ **Post date:** [June 18, 2021, 7:40am UTC](https://discuss.elastic.co/t/data-stream-data-retrieval-and-compression/276272/3 "2021-06-18T07:40:01Z")
> [@warkolm](#): > > Retrieving how exactly? Are the phases on the same hosts, or do you have different hardware profiles for each phase? They are on the same host only. I just want to know if the data retrieval on warm and cold phases have any difference in time it takes. I tried using a \_search query on an index in hot and warm phases on postman. I couldn't see much difference in getting the data. But for large data, how large will be the time lag on different phases ---
**Author:** ![prithvirajsivarajan](https://avatars.discourse-cdn.com/v4/letter/p/8e8cbc/32.png) [@prithvirajsivarajan](https://discuss.elastic.co/u/prithvirajsivarajan)\ **Post date:** [June 18, 2021, 7:41am UTC](https://discuss.elastic.co/t/data-stream-data-retrieval-and-compression/276272/4 "2021-06-18T07:41:33Z")
> [@warkolm](#): > > Elasticsearch compresses by default. You cannot zip the underlying data without losing access to it. So in cold phase, we can't reduce it's size much further using any compression technology? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 20, 2021, 11:32pm UTC](https://discuss.elastic.co/t/data-stream-data-retrieval-and-compression/276272/5 "2021-06-20T23:32:54Z")
If they are on the same host, no. > [@prithvirajsivarajan](#): > > So in cold phase, we can't reduce it's size much further using any compression technology? No. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 18, 2021, 11:33pm UTC](https://discuss.elastic.co/t/data-stream-data-retrieval-and-compression/276272/6 "2021-07-18T23:33:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.