# How can logstash support daily rolling kafka topics?

**URL:** <https://discuss.elastic.co/t/how-can-logstash-support-daily-rolling-kafka-topics/149209>\
**Category:** Logstash\
**Created:** [September 20, 2018, 1:36am UTC](https://discuss.elastic.co/t/how-can-logstash-support-daily-rolling-kafka-topics/149209 "2018-09-20T01:36:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ysn2233](https://avatars.discourse-cdn.com/v4/letter/y/838e76/32.png) [@ysn2233](https://discuss.elastic.co/u/ysn2233)\
**Post date:** [September 20, 2018, 1:36am UTC](https://discuss.elastic.co/t/how-can-logstash-support-daily-rolling-kafka-topics/149209/1 "2018-09-20T01:36:31Z")

</div>

This is a continuous conversation of [https://github.com/elastic/logstash/issues/10005](https://github.com/elastic/logstash/issues/10005)  
First I answer two questions repied by yaauie.

1. What we think is to use topics like ["xxx\_20180919", "xxx\_20180920", "xxx\_20180921"] and keep updating every day which can avoid the windows problem at 0:00 am.
2. We have used kafka config log.retention.hours but it only clean the data of topics instead of removing the topics themselves, which still require us to delete the topic manually.

Mentioned before, if we use logstash topics\_pattern like xxx\_.\*, the logstash will stuck if we delete a old topic and requires restarting. Does any one has an idea for our cases? we just want the logstash to consumer the kafka topics related to yesterday, today and tomorrow and updating every day.

Thank you very much!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 18, 2018, 1:36am UTC](https://discuss.elastic.co/t/how-can-logstash-support-daily-rolling-kafka-topics/149209/2 "2018-10-18T01:36:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
