# How to create index as per file creation date?

**URL:** <https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206>\
**Category:** Logstash\
**Created:** [October 3, 2019, 3:34pm UTC](https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206 "2019-10-03T15:34:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [October 3, 2019, 3:34pm UTC](https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206/1 "2019-10-03T15:34:50Z")

</div>

Hi Experts ,

For now I am creating index in ES on daily basis with below logic

> output {
> 
> ```
> elasticsearch {
> action => "index"
> hosts => ["elastic:1234"]
> index => "test-%{+YYYY.MM.dd}"
> document_type => "abc"
> }
> 
> ```
> 
> }

Unfortunately from 30th sep my LS service was down and below files are still waiting to get indexed . If I start LS today (03-10-2019)it will parse and save data in today's Index which I do not want . I want to create index as per file date . I have files like below  
30\_09\_2019\_test.log  
01\_10\_2019\_test.log  
02\_10\_2019\_test.log  
03\_10\_2019\_test.log

And I want to create index as below  
test-2019.09.30  
test-2019.10.01  
test-2019.10.02

I hope I was clear with what I want to achieve . Please let me know if you need further info which I can provide .

Regards  
VG

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 3, 2019, 5:52pm UTC](https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206/2 "2019-10-03T17:52:40Z")

</div>

"test-%{+YYYY.MM.dd}" is evaluated using the [@timestamp] field. You need to use a [date](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html) filter to parse a field that contains the date of the events to set [@timestamp].

---

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [October 3, 2019, 7:03pm UTC](https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206/3 "2019-10-03T19:03:00Z")

</div>

Well this is exactly what I was thinking but here is the challenge that I have the date filter that I have rt now is on another field and I am not treating this as timestamp field . Here is what I have

`date {match => ["managerReceiptTime","dd MMM YYYY HH:mm:ss 'IST'"] target => "managerReceiptTime"}`

and at kibana level I am defining index pattern based on this date . So is there any way that "test-%{+YYYY.MM.dd}" settings picks date from "managerReceiptTime" or from file name than timestamp field . If not then I have to convert managerReceiptTime and target to timestamp which I do not want as things will be disturbed . Can you suggest any other workaround ?

VG

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 3, 2019, 7:35pm UTC](https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206/4 "2019-10-03T19:35:00Z")

</div>

> [@vikas\_gopal](#):
>
> is there any way that "test-%{+YYYY.MM.dd}" settings picks date from "managerReceiptTime" or from file name

You could build the index name in a ruby filter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 31, 2019, 7:35pm UTC](https://discuss.elastic.co/t/how-to-create-index-as-per-file-creation-date/202206/5 "2019-10-31T19:35:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.