Logstash-output-file-as time based

[jawad846 (Mohamed Jawad)]

Hi Folks,

My output of logstash directed to the file called apache.log. This file needs to be generated in every hour.

For Example: apache-2018-04-16-10:00.log or something similar to this.

Here my configuration file :

# INPUT HERE
input {
    beats {
          port => 5044
    }
}

# FILTER HERE
filter {
    if [source]=="/var/log/apache2/error.log"
    {
        mutate {
            remove_tag => [ "beats_input_codec_plain_applied" ]
            add_tag => [ "apache_logs" ]
        }
    }
    if [source]=="/var/log/apache2/access.log"
    {
        mutate {
            remove_tag => [ "beats_input_codec_plain_applied" ]
            add_tag => [ "apache_logs" ]
        }
    }
}

# OUTPUT HERE
output {
    if "apache_logs" in [tags] {
        file {
            path => "/home/ubuntu/apache/apache-%{+yyyy-mm-dd}.log"
                codec => "json"
        }
    }
}

Please help out to solve.

@magnusbaeck

[guyboertje]

Looking at File output plugin | Logstash Reference [8.11] | Elastic
This in particular

One may also utilize the path option for date-based log rotation via the joda time format

Googling I find DateTimeFormat (Joda-Time 2.12.5 API)
From this I would try:

path => "/home/ubuntu/apache/apache-%{+yyyy-mm-dd-H}:00.log"`

[jawad846 (Mohamed Jawad)]

@guyboertje

path => "/home/ubuntu/apache/apache-%{+yyyy-mm-dd-H}:00.log"`

am getting the output for this syntax:

apache-2018-20-18-6:00.log
apache-2018-21-18-6:00.log

current date and time is : Wed Apr 18 11:53:22 IST 2018

@guyboertje

[jawad846 (Mohamed Jawad)]

Its work fine

# OUTPUT HERE
output {
if "apache_logs" in [tags] {
file {
path => "/home/ubuntu/apache/%{+YYYY}/%{+MM}/%{+dd}/%{+HH}/apache-%{host}-%{+YYYY-MM-dd-HH-zz}.log"
codec => "json"
}
}
}

files:

root@ip-192-168-2-79:/home/ubuntu/apache/2018/04/18/07# ls
apache-ip-192-168-2-196-2018-04-18-07-UTC.log  
apache-ip-192-168-2-223-2018-04-18-07-UTC.log

thanks @guyboertje