# How many months of log data do we retain in Elasticsearch? **URL:** \ **Category:** Elasticsearch\ **Created:** [August 6, 2019, 9:11pm UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104 "2019-08-06T21:11:14Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Bjen\_Shah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bjen_shah/32/46250_2.png) [@Bjen\_Shah](https://discuss.elastic.co/u/Bjen_Shah)\ **Post date:** [August 6, 2019, 9:11pm UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104/1 "2019-08-06T21:11:14Z")
How many months of log data does organization should store in Elasticsearch? Do we store the log data anywhere aside from Elasticsearch (e.g. flat files?) How do we archive old log data that is required for regulatory compliance but not needed “online” in the Elasticsearch cluster? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 7, 2019, 7:24am UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104/2 "2019-08-07T07:24:33Z")
The usual answer here is 'it depends'. First, the amount of data to retain depends on your budget, your hardware and potentially also in legal requirements of the country where the data is stored or processed. Especially sometimes in the latter case you dont have a choice. Regarding archiving, you might want to check out [Index Lifecycle Management](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/index-lifecycle-management.html) and soon snapshot lifecycle management (you should take a look at snapshots as well). Also [frozen indices](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/frozen-indices.html) are helpful in this case. ---
**Author:** ![Bjen\_Shah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bjen_shah/32/46250_2.png) [@Bjen\_Shah](https://discuss.elastic.co/u/Bjen_Shah)\ **Post date:** [August 7, 2019, 2:41pm UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104/3 "2019-08-07T14:41:12Z")
Thanks for your answer, actually I am looking to find out what's best practices of other company's. Now I am throwing another question ? - Can we have SOAP logs retention for week and others for more time as long we needed. ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 8, 2019, 9:36am UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104/4 "2019-08-08T09:36:08Z")
regarding your other question: easiest would be to store your SOAP logs in a dedicated index and use ILM to automatically delete those indices (or do it manually) after a different time period compared to the other indices. ---
**Author:** ![Bjen\_Shah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bjen_shah/32/46250_2.png) [@Bjen\_Shah](https://discuss.elastic.co/u/Bjen_Shah)\ **Post date:** [August 9, 2019, 8:30pm UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104/5 "2019-08-09T20:30:28Z")
Dear Spinscale, can you attach web link of ILM, which you have mentioned for reply of my other question. is there other tools available beside curator for archiving data or not ? when we archive the data or back up what's best format to compress the data or do we have some tools to encrypt them as well or not ? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 12, 2019, 7:53am UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104/6 "2019-08-12T07:53:43Z")
ILM refers to Index Lifecycle Management, which I linked in the above post already - sorry for the confusion. ---
**Author:** ![admlko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/admlko/32/21787_2.png) [@admlko](https://discuss.elastic.co/u/admlko)\ **Post date:** [August 12, 2019, 8:01am UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104/7 "2019-08-12T08:01:44Z")
> is there other tools available beside curator for archiving data or not ? > when we archive the data or back up what's best format to compress the data or do we have some tools to encrypt them as well or not ? Check out elasticdump: > **[GitHub - elasticsearch-dump/elasticsearch-dump: Import and export tools for...](https://github.com/elasticsearch-dump/elasticsearch-dump)** > > Import and export tools for elasticsearch & opensearch - GitHub - elasticsearch-dump/elasticsearch-dump: Import and export tools for elasticsearch & opensearch ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 9, 2019, 8:01am UTC](https://discuss.elastic.co/t/how-many-months-of-log-data-do-we-retain-in-elasticsearch/194104/8 "2019-09-09T08:01:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.