# Logstash output file **URL:** \ **Category:** Elasticsearch\ **Created:** [October 8, 2015, 5:34am UTC](https://discuss.elastic.co/t/logstash-output-file/31817 "2015-10-08T05:34:03Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Navneet\_Mathpal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/navneet_mathpal/32/3677_2.png) [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Post date:** [October 8, 2015, 5:34am UTC](https://discuss.elastic.co/t/logstash-output-file/31817/1 "2015-10-08T05:34:03Z")
Hi , I have a problem where I am getting some logs along with ID , I have two es Index (archive\_index , and latest\_logs) . so each time I have to check whether the current ID is same as previous ID if yes I will send the logs to my latest\_logs , if not all the logs which are in latest\_logs will go archive into archive index. I thought archiving the logs can not be done using logstash. (Because for archiving first I have to check the ID , AND then getting all the records from archive\_index then push them on latest\_logs and then delete all the records from latest\_logs for getting only new ID logs). So I am correctly doing like this logstash -\> file -\> java -\> es But the problem is when I am writing all the logs into file , file size is getting increase very rapidly (some G.B in few days). SO I want the alternative solution for that . 1. Is it possible from logstash I will feed my logs to kafta then kafta to java ? (if yes how will kafta ensure that after this much amount of data previous data should flush (if java has already got that )). Best Regards, Navneet ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 8, 2015, 5:53am UTC](https://discuss.elastic.co/t/logstash-output-file/31817/2 "2015-10-08T05:53:26Z")
If your goal is to have one index per ID, how about including the ID in the index name? If you need to be able to refer to the current ID via a symbolic "latest" name, set up an Elasticsearch alias for that. I don't think you can get Logstash to do that though. Similarly, if you want to be able to refer to non-current indexes via a symbolic "archive" name you can use an alias for that too. ---
**Author:** ![Navneet\_Mathpal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/navneet_mathpal/32/3677_2.png) [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Post date:** [October 8, 2015, 7:05am UTC](https://discuss.elastic.co/t/logstash-output-file/31817/3 "2015-10-08T07:05:39Z")
scenario is that -I will be having two index only. So only logs having latest ID will be in latest\_index (for that I have to check whether the current ID is same as previous IF yes then it means that id has not changed and it is latest ID so I will push my doc into latest\_index , if ID gets changed then all the doc are in latest\_index will go archive . In archive index there can have many IDs but in the latest\_index there will have logs with current running ID.) EX - if am getting 10 IDs , 9 IDs along with their logs would be in archive\_index and one latest ID along with its log will be in latest\_index ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 8, 2015, 7:47am UTC](https://discuss.elastic.co/t/logstash-output-file/31817/4 "2015-10-08T07:47:50Z")
Yes, the design is clear but I think it's flawed and will lead to unnecessary complexity to can be avoided with index aliases. What isn't clear is why it's so important to have a "latest" index. Whatever problem that design is trying to address could have other solutions. ---
**Author:** ![Navneet\_Mathpal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/navneet_mathpal/32/3677_2.png) [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Post date:** [October 8, 2015, 8:56am UTC](https://discuss.elastic.co/t/logstash-output-file/31817/5 "2015-10-08T08:56:16Z")
how does aliasing the index can solve the problem ? I have two kibana dashboards one points out to latest\_id and another points to archive\_id. Is their another solution possible ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 8, 2015, 9:40am UTC](https://discuss.elastic.co/t/logstash-output-file/31817/6 "2015-10-08T09:40:56Z")
An index alias "latest\_id" that points to the currently latest index allows you to keep your existing Kibana dashboard without changes and have ES transparently translate all requests for "latest\_id" to the current index. Same thing with "archive\_id" except that it can point to multiple indexes (all indexes except the latest one). _If_ you at some point want to consolidate the archive indexes into one big index (or a few larger ones) you can do that and update the alias accordingly. This can even be an atomic operation. 1. Copy documents from physical indexes archive\_id\_1, archive\_id\_2, and archive\_id\_3 to archive\_id\_123. 2. Reconfigure the archive\_id alias to point to archive\_id\_123. 3. Delete archive\_id\_1, archive\_id\_2, and archive\_id\_3. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:45pm UTC](https://discuss.elastic.co/t/logstash-output-file/31817/7 "2017-07-05T23:45:55Z")