Hi @guyboertje,
As suggested here is what I did...
Enabled Trace on on a few loggers.
curl -XPUT 'http://myserver.com:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d'
{
"logger.filewatch.discoverer": "TRACE",
"logger.filewatch.observingtail": "TRACE",
"logger.filewatch.sincedbcollection": "TRACE",
"logger.filewatch.tailmode.handlers.createinitial": "TRACE",
"logger.filewatch.tailmode.handlers.grow": "TRACE",
"logger.filewatch.tailmode.handlers.timeout": "TRACE",
"logger.filewatch.tailmode.processor": "TRACE",
"logger.filewatch.tailmode.handlers.create": "TRACE",
"logger.filewatch.tailmode.handlers.shrink": "TRACE",
"logger.logstash.filters.date": "TRACE",
"logger.logstash.filters.drop": "TRACE",
"logger.logstash.filters.grok": "TRACE",
"logger.logstash.inputs.file": "TRACE"
}
'
Output from my logger API
{
"host": "myserver.com",
"version": "6.6.0",
"http_address": "mysever.com:9600",
"id": "44219357-5abf-4028-aa44-f955ded1d33b",
"name": "myserver.com",
"loggers": {
"filewatch.discoverer": "TRACE",
"filewatch.observingtail": "TRACE",
"filewatch.sincedbcollection": "TRACE",
"filewatch.tailmode.handlers.create": "TRACE",
"filewatch.tailmode.handlers.createinitial": "TRACE",
"filewatch.tailmode.handlers.grow": "TRACE",
"filewatch.tailmode.handlers.shrink": "TRACE",
"filewatch.tailmode.processor": "TRACE",
"logstash.agent": "DEBUG",
"logstash.api.service": "DEBUG",
"logstash.codecs.plain": "DEBUG",
"logstash.config.source.local.configpathloader": "DEBUG",
"logstash.config.source.multilocal": "DEBUG",
"logstash.config.sourceloader": "DEBUG",
"logstash.configmanagement.extension": "DEBUG",
"logstash.filters.date": "TRACE",
"logstash.filters.drop": "TRACE",
"logstash.filters.grok": "TRACE",
"logstash.filters.kv": "DEBUG",
"logstash.filters.mutate": "DEBUG",
"logstash.inputs.file": "TRACE",
"logstash.instrument.periodicpoller.cgroup": "DEBUG",
"logstash.instrument.periodicpoller.deadletterqueue": "DEBUG",
"logstash.instrument.periodicpoller.jvm": "DEBUG",
"logstash.instrument.periodicpoller.os": "DEBUG",
"logstash.instrument.periodicpoller.persistentqueue": "DEBUG",
"logstash.modules.scaffold": "DEBUG",
"logstash.modules.xpackscaffold": "DEBUG",
"logstash.monitoringextension": "DEBUG",
"logstash.monitoringextension.pipelineregisterhook": "DEBUG",
"logstash.outputs.elasticsearch": "DEBUG",
"logstash.pipeline": "DEBUG",
"logstash.plugins.registry": "DEBUG",
"logstash.runner": "DEBUG",
"logstash.setting.writabledirectory": "DEBUG",
"logstash.util.decorators": "DEBUG",
"org.logstash.Event": "DEBUG",
"org.logstash.FieldReference": "DEBUG",
"org.logstash.Logstash": "DEBUG",
"org.logstash.execution.AbstractPipelineExt": "DEBUG",
"org.logstash.execution.ShutdownWatcherExt": "DEBUG",
"org.logstash.execution.WorkerLoop": "DEBUG",
"org.logstash.filters.DateFilter": "DEBUG",
"org.logstash.instrument.metrics.gauge.LazyDelegatingGauge": "DEBUG",
"org.logstash.instrument.monitors.HotThreadsMonitor": "DEBUG",
"org.logstash.plugins.pipeline.PipelineBus": "DEBUG",
"org.logstash.secret.store.SecretStoreFactory": "DEBUG",
"slowlog.logstash.codecs.plain": "TRACE",
"slowlog.logstash.filters.date": "TRACE",
"slowlog.logstash.filters.drop": "TRACE",
"slowlog.logstash.filters.grok": "TRACE",
"slowlog.logstash.filters.kv": "TRACE",
"slowlog.logstash.filters.mutate": "TRACE",
"slowlog.logstash.inputs.file": "TRACE",
"slowlog.logstash.outputs.elasticsearch": "TRACE"
}
}
Logs are not showing much the same messages are still being shown.
[2019-04-10T16:00:07,690][DEBUG][logstash.api.service ] [api-service] start
[2019-04-10T16:00:07,690][DEBUG][logstash.api.service ] [api-service] start
[2019-04-10T16:00:07,691][DEBUG][logstash.api.service ] [api-service] start
[2019-04-10T16:00:07,691][DEBUG][logstash.api.service ] [api-service] start
[2019-04-10T16:00:07,691][DEBUG][logstash.api.service ] [api-service] start
[2019-04-10T16:00:07,747][DEBUG][logstash.agent ] API HTTP Request {:status=>200, :request_method=>"PUT", :path_info=>"/_node/logging", :query_string=>"pretty", :http_version=>"HTTP/1.1", :http_accept=>"/"}
[2019-04-10T16:00:10,689][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2019-04-10T16:00:10,689][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2019-04-10T16:00:11,201][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[2019-04-10T16:00:15,804][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2019-04-10T16:00:15,804][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2019-04-10T16:00:16,209][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[2019-04-10T16:00:20,941][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2019-04-10T16:00:20,941][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2019-04-10T16:00:21,214][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[2019-04-10T16:00:26,061][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2019-04-10T16:00:26,097][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2019-04-10T16:00:26,315][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup,
Please let me know if you see anything that stands out. Once again thanks for taking the time to look into this issue.