Logstash doesn’t process / cgroup files

Elastic Stack Logstash
1

Dear all

I installed logstash 6.6.0 and ran a first sequence that picks up a csv file and stdouts on the screen. It worked as expected. Then I added and removed a translate filter. Both times logstash didn’t start processing. In the debug mode it outputs the message below repeatedly.
The problem seems to be known but there I find no solution. There is a .rb file in logstash that contains the DIR paths.
What easy way is there to solve? How important are these cgroups - can I just eg delete the paths and it is fixed?

Best Michael

[DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu

2

IIRC, cgroups are only relevant or important when using LS in a docker container as the CPU metrics are sourced from them.

If you are not using docker then you can ignore them and switch to logging at INFO.

FYI, you can enable very targeted debug logging by turning it on for very specific subsystems via the LS REST API.

The logging API allows for different levels of logging for different components in LS.

First do curl -XGET 'localhost:9600/_node/logging?pretty'
You see something like this:

{
  "host" : "Elastics-MacBook-Pro.local",
  "version" : "6.4.0",
  "http_address" : "127.0.0.1:9600",
  "id" : "8789409b-7126-4034-9347-de47e6ce12a9",
  "name" : "Elastics-MacBook-Pro.local",
  "loggers" : {
    "filewatch.discoverer" : "INFO",
    "filewatch.observingtail" : "INFO",
    "filewatch.sincedbcollection" : "INFO",
    "filewatch.tailmode.handlers.createinitial" : "INFO",
    "filewatch.tailmode.processor" : "INFO",
    "logstash.agent" : "INFO",
    "logstash.api.service" : "INFO",
    "logstash.codecs.json" : "INFO",
    ...
    "logstash.filters.grok" : "INFO",
    "logstash.filters.date" : "INFO",
    "logstash.inputs.file" : "INFO",
    ...
    "logstash.outputs.stdout" : "INFO",
    "logstash.pipeline" : "INFO",
    ...
    "slowlog.logstash.codecs.json" : "INFO",
    "slowlog.logstash.codecs.rubydebug" : "INFO",
    "slowlog.logstash.filters.date" : "INFO",
    "slowlog.logstash.inputs.file" : "INFO",
    "slowlog.logstash.outputs.stdout" : "INFO"
  }
}

Using the API
Turn DEBUG on for just the date and grok filters:

curl -XPUT 'localhost:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d'
{
    "logstash.filters.date" : "DEBUG"
    "logstash.filters.grok" : "DEBUG"
}
'

Turn trace off:

curl -XPUT 'localhost:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d'
{
    "logstash.filters.date" : "WARN"
    "logstash.filters.grok" : "WARN"
}
'

Or

curl -XPUT 'localhost:9600/_node/logging/reset?pretty'

NOTE: it might be a good idea to start LS with logging set to WARN in the logstash.yml so other logging is less verbose.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.