# \[FILEBEAT\] Insert timestamp inside of logfile name

**URL:** <https://discuss.elastic.co/t/filebeat-insert-timestamp-inside-of-logfile-name/249843>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 24, 2020, 3:03pm UTC](https://discuss.elastic.co/t/filebeat-insert-timestamp-inside-of-logfile-name/249843 "2020-09-24T15:03:57Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mstojanovic](https://avatars.discourse-cdn.com/v4/letter/m/dc4da7/32.png) [@mstojanovic](https://discuss.elastic.co/u/mstojanovic)\
**Post date:** [September 24, 2020, 3:03pm UTC](https://discuss.elastic.co/t/filebeat-insert-timestamp-inside-of-logfile-name/249843/1 "2020-09-24T15:03:58Z")

</div>

Hello,

I want to gather logs from a specific location. As the log's name is changed each day, I would like to insert timestamp in log file name. By doing so, each new day, the correct log name will be parsed by filebeat.

Log file name examples:

```auto
repository.cleanup-*20200921*010000083.log
repository.cleanup-*20200922*010000095.log
repository.cleanup-*20200923*010000097.log
repository.cleanup-*20200924*010000067.log

```

**The text between stars in the file name I need to replace with a custom timestamp.** The rest of the logfile name will be done by regex.

This is my input section:

```auto
- type: log
  enabled: true
  paths:
    - /opt/sonatype-work/nexus3/log/tasks/repository.cleanup-{SOME TIMESTAMP HERE}01{SOME REGEX}.log
  fields:
    kafka_topic: nexus-repository-cleanup-logs

```

Would be grateful for any suggestions!  
Thank you.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 25, 2020, 2:17pm UTC](https://discuss.elastic.co/t/filebeat-insert-timestamp-inside-of-logfile-name/249843/2 "2020-09-25T14:17:01Z")

</div>

Why do you need a special regex for the timestamp? Why not just use `/opt/sonatype-work/nexus3/log/tasks/repository.cleanup-[0-9]{8}01[0-9]{6}.log`?

---

<div class="post-metadata">

**Author:** ![mstojanovic](https://avatars.discourse-cdn.com/v4/letter/m/dc4da7/32.png) [@mstojanovic](https://discuss.elastic.co/u/mstojanovic)\
**Post date:** [September 25, 2020, 3:15pm UTC](https://discuss.elastic.co/t/filebeat-insert-timestamp-inside-of-logfile-name/249843/3 "2020-09-25T15:15:50Z")

</div>

Hello Noemi!

Well, the directory is full of old logs. And when I use a regex that can match any character like [0-9]{8}01[0-9]{6}, filebeat will go through all old logs and send their data to Kafka and so own to Elastic ( as old logs are not being zipped ).

So, to evade that and a sideway solution which would be to delete all old logs so it can start with the current active log, I have asked is it possible to use a timestamp in log name?

Also, another thing is, that, when logs get deleted from the Kafka topic after 7 days of retention, filebeat will instantly again parse those old logs and send them again to Kafka.

And when logs get deleted from the index in Elastic after some retention period, Kafka would also send those old logs again to Elastic. That will repeat constantly unless we setup some cron job that will delete logs on the server. But all of this is a not proper solution.

And in my opinion, best would be the timestamp in logfile name? Is that possible, and if so, do you have a suggestion?

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 23, 2020, 5:15pm UTC](https://discuss.elastic.co/t/filebeat-insert-timestamp-inside-of-logfile-name/249843/4 "2020-10-23T17:15:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.