Filebeat 7.11.1 registry log.json growing wild

Elastic Stack Beats
,
1

Hi,

I just migrate from filebeat 7.8 (installed in host os) to filebeat 7.11 (installed in docker).
I bind the volume in docker-compose for the registry and log files to be processed:

    version: "2.2"
services:
  filebeat:
    image: docker.elastic.co/beats/filebeat:7.11.1
    container_name: filebeat
    restart: unless-stopped
    hostname: BJBMWJAVAAPPDEV
    # Need to override user so we can access the log files, and docker.sock
    user: root
    volumes:
      - "./filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro"
      - "./filebeat/fields.yml:/usr/share/filebeat/fields.yml:ro"
      # this is where filebeat keep the offset of each file being processed
      - "./filebeat/registry:/usr/share/filebeat/data/registry/filebeat:rw"
      # this is where log files to be processed
      - "/opt/hanoman/dev-java/log/:/usr/share/filebeat/log:ro"
      - "/var/lib/docker/containers:/hostfs/var/lib/docker/containers:ro"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    command: filebeat -e -strict.perms=false

The filebeat.yml is something like this:

    - type: log
  enabled: true
  paths:
    - /usr/share/filebeat/log/*.log
  json.keys_under_root: true
  json.add_error_key: true
  json.overwrite_keys: true

  clean_removed: true

# ============================== Filebeat modules ==============================

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
  #reload.period: 10s

# ------------------------------ Logstash Output -------------------------------
output.logstash:
  hosts: ["10.6.226.80:5050"]
  protocol: http

logging.level: info
logging.metrics.enabled: false

I noticed a few things:

  1. in ./filebeat/registry there is a file log.json which grow really fast. I did not find this file in the previous installation of Filebeat 7.8. The content of the file is something like this:
    {"k":"filebeat::logs::native::100822905-64768","v":{"prev_id":"","source":"/usr/share/filebeat/log/tcpgw-core2.2021-02-17_14.log","FileStateOS":{"inode":100822905,"device":64768},"id":"native::100822905-64768","offset":20345574,"timestamp":[1804868356216,1613905762],"ttl":-1,"type":"log","identifier_name":"native"}}
{"op":"set","id":8802395}
{"k":"filebeat::logs::native::101040454-64768","v":{"id":"native::101040454-64768","prev_id":"","source":"/usr/share/filebeat/log/tcpgw-http-server1.2021-02-18_15.log","offset":12057,"timestamp":[1803903103588,1613905763],"ttl":-1,"type":"log","FileStateOS":{"device":64768,"inode":101040454},"identifier_name":"native"}}
{"op":"set","id":8802396}
{"k":"filebeat::logs::native::102656345-64768","v":{"id":"native::102656345-64768","prev_id":"","identifier_name":"native","source":"/usr/share/filebeat/log/tcpgw-core1.2021-01-28_14.log","offset":2999545,"timestamp":[1803953777445,1613905763],"ttl":-1,"type":"log","FileStateOS":{"inode":102656345,"device":64768}}}

How to prevent Filebeat from producing this log file?

  1. There is data.json in previous Filebeat 7.8, but I could not find this file in Filebeat 7.11.1. Instead there is a file with {some_numbers}.json. The content of the file is something like this:
    {"_key":"filebeat::logs::native::102466164-64768","id":"native::102466164-64768","source":"/usr/share/filebeat/log/tcpgw-core2.2021-02-02_03.log","ttl":-1,"FileStateOS":{"inode":102466164,"device":64768},"identifier_name":"native","prev_id":"","offset":1314816,"timestamp":[1804600953615,1613905749],"type":"log"},
{"_key":"filebeat::logs::native::102702249-64768","prev_id":"","source":"/usr/share/filebeat/log/tcpgw-core1.2021-02-02_05.log","ttl":-1,"identifier_name":"native","id":"native::102702249-64768","offset":1300656,"timestamp":[1804633383556,1613905749],"type":"log","FileStateOS":{"inode":102702249,"device":64768}},
{"_key":"filebeat::logs::native::102396165-64768","ttl":-1,"identifier_name":"native","prev_id":"","timestamp":[1804674825228,1613905749],"offset":21720,"type":"log","FileStateOS":{"inode":102396165,"device":64768},"id":"native::102396165-64768","source":"/usr/share/filebeat/log/tcpgw-http-server2.2021-02-12_02.log"},
{"_key":"filebeat::logs::native::102112568-64768","ttl":-1,"FileStateOS":{"inode":102112568,"device":64768},"prev_id":"","offset":6336,"timestamp":[1804713974899,1613905749],"identifier_name":"native","id":"native::102112568-64768","source":"/usr/share/filebeat/log/tcpgw-iso-server2.2021-01-27_10.log","type":"log"},
{"_key":"filebeat::logs::native::102941182-64768","id":"native::102941182-64768","timestamp":[1804749686182,1613905749],"ttl":-1,"FileStateOS":{"inode":102941182,"device":64768},"prev_id":"","source":"/usr/share/filebeat/log/tcpgw-echannel-iso8583-server2.2021-02-16_17.log","offset":85272,"type":"log","identifier_name":"native"}

is this the replacement of data.json in Filebeat 7.8 registry?
and why the some_number changes over time?

Please enlighten me.

Thank you.

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.