# Best method for daily ingest of data?

**URL:** <https://discuss.elastic.co/t/best-method-for-daily-ingest-of-data/111593>\
**Category:** Elasticsearch\
**Created:** [December 13, 2017, 3:40pm UTC](https://discuss.elastic.co/t/best-method-for-daily-ingest-of-data/111593 "2017-12-13T15:40:31Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![charles221](https://avatars.discourse-cdn.com/v4/letter/c/b5e925/32.png) [@charles221](https://discuss.elastic.co/u/charles221)\
**Post date:** [December 13, 2017, 3:40pm UTC](https://discuss.elastic.co/t/best-method-for-daily-ingest-of-data/111593/1 "2017-12-13T15:40:31Z")

</div>

Heya,

I'm setting up a Elasticsearch & Kibana instance aimed for analysing trends over time. Every day we collect from a few new sources, then ingest updated data for the established sources. The issue is that, at the moment, the daily ingest is overwriting the original data - when instead we would like the original data to remain so that we can analyse trends e.g. number of hits per day, changes in metadata, etc.

I feel like I am missing a basic technique in using Elasticsearch - would it be possible to index data with a shared identifier between ingests without deleting previous data?

Thank you for your time 🙂

---

<div class="post-metadata">

**Author:** ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)\
**Post date:** [December 13, 2017, 6:00pm UTC](https://discuss.elastic.co/t/best-method-for-daily-ingest-of-data/111593/2 "2017-12-13T18:00:28Z")

</div>

Elasticsearch supports date math in indices' names, so you can create daily indices, check out:

[https://www.elastic.co/guide/en/elasticsearch/reference/6.0/date-math-index-names.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.0/date-math-index-names.html)

Alternatively, you can set up the rollover API - [https://www.elastic.co/guide/en/elasticsearch/reference/6.0/indices-rollover-index.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.0/indices-rollover-index.html)

This will allow you to set up an alias for reading from, and an alias for writing to, and then periodically hit the API so that a new index will be created (so you don't overwrite yesterday's data with today's, for instance)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 10, 2018, 6:00pm UTC](https://discuss.elastic.co/t/best-method-for-daily-ingest-of-data/111593/3 "2018-01-10T18:00:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.