Hello,
Logstash starts successfully but doesn't generate events when I change path to include subdirectories.
My logstash.conf looks like:-
input {
file {
path => "/local/kdmlogs/2019031905/*/*"
exclude => "*.s3logs"
start_position => "beginning"
sincedb_path => "/tmp/.sincedb"
tags => ["logs"]
}
}filter {
if "logs" in [tags] {
grok {
match => {
"message" => "%{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:date}:%{HOUR}%{MINUTE}(%{SECOND}) %{SYSLOGPROG}:%{GREEDYDATA:syslog_message}"
}
}grok { match => { "path" => "/local/kdmlogs/2019031905/*/%{GREEDYDATA}#%{GREEDYDATA:device_type}#%{GREEDYDATA}#%{GREEDYDATA:dsn}#%{GREEDYDATA:s3key}" } } #Remove fields we don 't want (to save space) mutate { remove_field => ["message", "path", "host"] }
}
metrics {
meter => "events"
add_tag => "metric"
}
}output {
if "logs" in [tags] {
if ([year]) {
stdout{}
}
}
if "metric" in [tags] {file { path => "/local/kdmlogs/metric/test-metric.txt" codec => line { format => "rate: %{[events][rate_1m]}" } }
}
}
In the input, under file, when I change the path to :-
path => "/local/kdmlogs/2019022022/1/*"
i.e. all files under subdirector "1" under directory "2019022022", it works well.
But with the above logstash config, the logstash starts but doesn't generate any events. I remember doing the same thing some time back and it worked well previosuly.
When I add a --debug, all I see is:-
[DEBUG] 2019-03-20 12:32:41.365 [Ruby-0-Thread-4: :1] file - Starting flush cycle
[DEBUG] 2019-03-20 12:32:43.365 [Ruby-0-Thread-4: :1] file - Starting flush cycle
[DEBUG] 2019-03-20 12:32:44.234 [pool-3-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
Can someone please help me with same? I've tried almost all the solutions posted on the internet.
This is only observed in Logstash 6.4.0 and works well in logstash 6.2.4
Thanks,
Yogendra