Error running input: error receiving slowlog data: ERR unknown command `SLOWLOG`, with args beginning with: `GET`

Hi I am using redis filebeat with the following config. However the error Error running input: error receiving slowlog data: ERR unknown command SLOWLOG, with args beginning with: GET occurs very frequently.

Thanks for any suggestions!

# ref: https://github.com/elastic/cloud-on-k8s/blob/master/config/recipes/beats/filebeat_autodiscover.yaml
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: {{ include "elastic.name.beat.filebeat" . }}
  labels: {{- include "elastic.labels" . | nindent 4 }}
spec:
  type: filebeat
  version: {{ .Values.filebeat.version }}
  image: "elastic/filebeat:{{ .Values.filebeat.version }}"
  elasticsearchRef:
    name: {{ include "elastic.name.elasticsearch" . }}
  kibanaRef:
    name: {{ include "elastic.name.kibana" . }}
  config:
    filebeat:
      autodiscover:
        providers:
          - type: kubernetes
            host: ${HOSTNAME}
            include_annotations:
              - tom_filebeat_mode
            hints:
              enabled: true
              default_config:
                type: container
                paths:
                  - /var/log/containers/*${data.kubernetes.container.id}.log
            templates:
              {{- range $k, $v := .Values.filebeat.redis.modeSentinel }}
              - condition:
                  equals:
                    kubernetes.annotations.tom_filebeat_mode: {{ $v.release }}
                config:
                  # https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-redis.html
                  # and https://discuss.elastic.co/t/redis-slowlog-monitoring-with-eck-elastic-on-kubernetes/246377/2
                  - module: redis
                    log:
                      enabled: true
                      var.paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
                    slowlog:
                      enabled: true
                      var.hosts: ["${data.host}:${data.kubernetes.annotations.redis_port}"]
                      # https://www.elastic.co/blog/securely-manage-credentials-monitoring-kubernetes-workloads-autodiscovery
                      var.password: "this_is_a_password"
{{/*                      var.password: "${kubernetes.{{ $.Release.Namespace }}.{{ $v.release }}.redis-password}"*/}}
              {{- end }}

...

That part actually expands to the following (got by reading the spec of the CRD in k8s)

spec:
  config:
    filebeat:
      autodiscover:
        providers:
          - hints:
              default_config:
                paths:
                  - '/var/log/containers/*${data.kubernetes.container.id}.log'
                type: container
              enabled: true
            host: '${HOSTNAME}'
            include_annotations:
              - tom_filebeat_mode
            templates:
              - condition:
                  equals:
                    kubernetes.annotations.tom_filebeat_mode: tom-redis-cache
                config:
                  - log:
                      enabled: true
                      var.paths:
                        - >-
                          /var/log/containers/*${data.kubernetes.container.id}.log
                    module: redis
                    slowlog:
                      enabled: true
                      var.hosts:
                        - '${data.host}:${data.kubernetes.annotations.redis_port}'
                      var.password: '${kubernetes.default.tom-redis-cache.redis-password}'
              - condition:
                  equals:
                    kubernetes.annotations.tom_filebeat_mode: tom-redis-custom-code
                config:
                  - log:
                      enabled: true
                      var.paths:
                        - >-
                          /var/log/containers/*${data.kubernetes.container.id}.log
                    module: redis
                    slowlog:
                      enabled: true
                      var.hosts:
                        - '${data.host}:${data.kubernetes.annotations.redis_port}'
                      var.password: >-
                        ${kubernetes.default.tom-redis-custom-code.redis-password}
              - condition:
                  equals:
                    kubernetes.annotations.tom_filebeat_mode: tom-redis-bloom
                config:
                  - log:
                      enabled: true
                      var.paths:
                        - >-
                          /var/log/containers/*${data.kubernetes.container.id}.log
                    module: redis
                    slowlog:
                      enabled: true
                      var.hosts:
                        - '${data.host}:${data.kubernetes.annotations.redis_port}'
                      var.password: '${kubernetes.default.tom-redis-bloom.redis-password}'
            type: kubernetes
...

Oh I find the reason: I also try to get slowlog for sentinels... But they do not support this. Only master/replica supports SLOWLOG command.

1 Like