Logstash starts successfully but doesn't generate events


(Yogendra Hada) #1

Hello,

Logstash starts successfully but doesn't generate events when I change path to include subdirectories.

My logstash.conf looks like:-

input {
file {
path => "/local/kdmlogs/2019031905/*/*"
exclude => "*.s3logs"
start_position => "beginning"
sincedb_path => "/tmp/.sincedb"
tags => ["logs"]
}
}

filter {
if "logs" in [tags] {
grok {
match => {
"message" => "%{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:date}:%{HOUR}%{MINUTE}(%{SECOND}) %{SYSLOGPROG}:%{GREEDYDATA:syslog_message}"
}
}

grok {
  match => {
    "path" => "/local/kdmlogs/2019031905/*/%{GREEDYDATA}#%{GREEDYDATA:device_type}#%{GREEDYDATA}#%{GREEDYDATA:dsn}#%{GREEDYDATA:s3key}"
  }
}

#Remove fields we don 't want (to save space)
mutate {
  remove_field => ["message", "path", "host"]
}

}
metrics {
meter => "events"
add_tag => "metric"
}
}

output {
if "logs" in [tags] {
if ([year]) {
stdout{}
}
}
if "metric" in [tags] {

file {
  path => "/local/kdmlogs/metric/test-metric.txt"
  codec => line {
    format => "rate: %{[events][rate_1m]}"
  }
}

}
}

In the input, under file, when I change the path to :-
path => "/local/kdmlogs/2019022022/1/*"
i.e. all files under subdirector "1" under directory "2019022022", it works well.

But with the above logstash config, the logstash starts but doesn't generate any events. I remember doing the same thing some time back and it worked well previosuly.

When I add a --debug, all I see is:-
[DEBUG] 2019-03-20 12:32:41.365 [Ruby-0-Thread-4: :1] file - Starting flush cycle
[DEBUG] 2019-03-20 12:32:43.365 [Ruby-0-Thread-4: :1] file - Starting flush cycle
[DEBUG] 2019-03-20 12:32:44.234 [pool-3-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu

Can someone please help me with same? I've tried almost all the solutions posted on the internet.

Thanks,
Yogendra


(system) closed #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.