Writing the Filebeat Registry

Elastic Stack Beats
1

Hi,

I´m using filebeat 6.3.2 in a docker container.

Shipping Logs works, and it looks the registry /usr/share/filebeat/data/registry is updated (timestamp increases, length changes).

But I´m getting these errors every minute or so.

ERROR registrar/registrar.go:346 Writing of registry returned error: rename /usr/share/filebeat/data/registry.new /usr/share/filebeat/data/registry: no such file or directory. Continuing...

I can get rid of this error if I explicitly set the filename via

filebeat.registry_file: ${path.data}/myregistry

And the error only occurs if I mount a host volume to /usr/share/filebeat/data/, but I have checked that the directory /usr/share/filebeat/data/ is writable for the docker container and the files as well (and the container runs with user root).

Any idea what provokes the error above?

docker run:

#!/bin/bash
set -x
docker rm -v fb
docker run \
 --env "TZ=Europe/Berlin" \
 --hostname dwpmicrappe04 \
 --log-driver json-file \
 --log-opt max-size=10g \
 --log-opt max-file=2 \
 -v /var/lib/docker:/var/lib/docker \
 -v /var/run/docker.sock:/var/run/docker.sock \
 -v $(pwd)/filebeat.yml:/usr/share/filebeat/filebeat.yml \
 -v /appdata/container/filebeat/data/:/usr/share/filebeat/data/ \
  --name fb \
  --user 0 \
  -l co.elastic.logs/disable=true \
  via/docker.elastic.co/beats/filebeat:6.2.4

filebeat config

---
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      templates:
        - condition:
            contains.docker.container.image: logprod
          config:
            - type: log
              paths:
                - /var/lib/docker/containers/${data.docker.container.id}/*.log
              json.keys_under_root: true
              fields:
                level: info
                labels: ${data.docker.container.labels.label1}
name: dwpmicrappe04
fields:
  app: filebeat
  stage: e
  customer: dwpbank
logging.level: info
fields_under_root: true
processors:
- decode_json_fields:
    fields: ["log"]
- rename:
    fields:
      - from: "log"
        to: "fredfroehn"
- drop_fields:
    fields: ["input", "time", "host"]
filebeat.registry_file: ${path.data}/myregistry
output.console:
  enabled: false
  pretty: true
output.redis:
  enabled: true
  hosts: ["2.104.19.25:6379", "2.104.19.26:6379", "2.104.19.27:6379", "2.104.19.28:6379"]
  key: logstash
  ssl.enabled: true
  ssl.verification_mode: none
logging.to_files: false
2

I have seen a similar issue before on our CI, but I could not reproduce it. https://github.com/elastic/beats/issues/7687
I am not sure yet if the root cause is the same, so please open a separate issue on Github. At least this problem can be reproduced consistently. https://github.com/elastic/beats/issues/new

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.