Dynamic input file name

Elastic Stack Logstash
1

Hi,

I have log file whose name changes as per date. How to set path in the input filter. i have given below path:

"/www/redhat/ews-abc-uat/tomcat7/logs/access.date '+%F'.log"

Its not working. Please suggest.

Regards,
Raman

2

That kind of filename pattern isn't supported. Why not just use /www/redhat/ews-abc-uat/tomcat7/logs/access.*.log?

3

Thanks for the reply. There are many files under logs directory and i want to process only current file not all. Is there any other option so that it will pick up only current date file ?

Regards,
Raman

4

There are many files under logs directory and i want to process only current file not all.

Are you having actual issues with the number of files that Logstash needs to keep open?

Is there any other option so that it will pick up only current date file ?

Short of periodically generating a Logstash configuration file with the currently correct filename pattern, I don't believe there is a way.

5

Hi Magnus,

I am facing the same issue in logstash 2.1.

Can we get sysdate inside input {} in configuration file.
If we can have sysdate, then we should create dynamic path using sysdate inside input {}. I think this is the only way.

Please let us know if it is possible.

6

For me it look like a great use case for config file reload (new since 2.3):
https://www.elastic.co/guide/en/logstash/current/reloading-config.html

Just rewrite config file with the current filename after rotating logs.

7

Hi,

Thanks for the suggestion. I really appreciate the effort.
It's nice concept to reload the config without stopping logstash.

But let me tell you, in my case a bunch of new log files generated every 24 hours. It will not be possible to change the config file every 24 hours and hit config reload. You know it's not a kind of automated way. We need to feed something to logstash everyday.

8

But let me tell you, in my case a bunch of new log files generated every 24 hours. It will not be possible to change the config file every 24 hours and hit config reload. You know it's not a kind of automated way. We need to feed something to logstash everyday.

It's not clear why this can't be automated. It's also not clear why you need to specify an exact filename rather than using "*.log" or some other filename pattern that matches all log files, past an future. Logstash will pick up newly created files with matching names within seconds.

9

Please refer below post. You'll get entire problem. Need your assistance.

Thanks in advance, Magnus !