# Filebeat cannot find registry file (/var/lib/filebeat/registry/filebeat) **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 14, 2020, 1:34pm UTC](https://discuss.elastic.co/t/filebeat-cannot-find-registry-file-var-lib-filebeat-registry-filebeat/214997 "2020-01-14T13:34:23Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![ntran](https://avatars.discourse-cdn.com/v4/letter/n/73ab20/32.png) [@ntran](https://discuss.elastic.co/u/ntran)\ **Post date:** [January 14, 2020, 1:34pm UTC](https://discuss.elastic.co/t/filebeat-cannot-find-registry-file-var-lib-filebeat-registry-filebeat/214997/1 "2020-01-14T13:34:23Z")
Hi all, I am getting this error message when I run filebeat -e: `Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory` I do not have a `/var/lib/filebeat/registry/filebeat/data.json.new` file, however, I do have `/var/lib/filebeat/registry/filebeat/data.json` Filebeat is still harvesting files, however, how does it know it has finished harvesting the file if it cannot find the registry file? > [root@elastic-nfs01 ~]# ls -lh /var/lib/filebeat/registry/filebeat > -rw-------. 1 root root 38K Jan 14 13:23 data.json > -rwxrwxrwx. 1 root root 16 Jul 24 07:17 meta.json Filebeat.yml #=========================== Filebeat inputs ============================= ``` filebeat.inputs: - type: log ignore_older: 48h clean_inactive: 72h clean_removed: true filebeat.registry.path: /var/lib/filebeat/registry # Change to true to enable this input configuration. # enabled: false paths: - /data/elastic-backup/logs/elastic-*/elastic-01_server.json - type: log # Change to true to enable this input configuration. # enabled: false paths: - /data/cluster-elastic-backup/logs/cluster-elastic*/cluster-cluster-01_server.json ignore_older: 48h clean_inactive: 72h clean_removed: true filebeat.registry.path: /var/lib/filebeat/registry #============================= Filebeat modules =============================== filebeat.config.modules: path: /etc/filebeat/modules.d/*.yml reload.enabled: false #==================== Elasticsearch template setting ========================== setup.template.settings: index.number_of_shards: 1 #============================== Kibana ===================================== setup.kibana: host: "https://XX.XX.XX.XX:5601" ssl.verification_mode: "none" #================================ Outputs ===================================== #-------------------------- Elasticsearch output ------------------------------ output.elasticsearch: # Array of hosts to connect to. hosts: ["XX.XX.XX.XX:9200"] # Optional protocol and basic auth credentials. protocol: "https" username: "elastic" password: "changeme" ssl.verification_mode: "none" bulk_max_size: 50 compression_level: 1 worker: 2 #================================ Processors ===================================== # Configure processors to enhance or manipulate events generated by the beat. processors: - add_host_metadata: ~ - add_cloud_metadata: ~ #================================ Logging ===================================== # Sets log level. The default log level is info. # Available log levels are: error, warning, info, debug #logging.level: debug # At debug level, you can selectively enable logging only for some components. # To enable all selectors use ["*"]. Examples of other selectors are "beat", # "publish", "service". #logging.selectors: ["*"] ``` Elasticsearch module > ``` > > # Module: elasticsearch > > # Docs: https://www.elastic.co/guide/en/beats/filebeat/7.2/filebeat-module-elasticsearch.html > > - module: elasticsearch > > # Server log > > server: > > enabled: false > > # Set custom paths for the log files. If left empty, > > # Filebeat will choose the paths depending on your OS. > > var.paths: > > - /data/elastic-backup/logs/elastic-hotdata01/*server.json > > # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1. > > var.convert_timezone: true > > audit: > > enabled: true > > # Set custom paths for the log files. If left empty, > > # Filebeat will choose the paths depending on your OS. > > var.paths: > > - /data/elastic-backup/logs/elastic-warmdata03/elastic-01_audit*.json > > - /data/elastic-backup/logs/elastic-hotdata01/elastic-01_audit*.log > > - > > # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1. > > var.convert_timezone: true > > ``` ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [January 14, 2020, 7:29pm UTC](https://discuss.elastic.co/t/filebeat-cannot-find-registry-file-var-lib-filebeat-registry-filebeat/214997/2 "2020-01-14T19:29:34Z")
> [@ntran](#): > > [root@elastic-nfs01 ~]# ls -lh /var/lib/filebeat/registry/filebeat > -rw-------. 1 root root 38K Jan 14 13:23 data.json > -rwxrwxrwx. 1 root root 16 Jul 24 07:17 meta.json Did you run this command before or after your started Filebeat? If you restart Filebeat now, do you still get the same error? Would you mind posting the exact Filebeat logs showing the error message? Finally, what version of Filebeat are you running? Thanks! ---
**Author:** ![ntran](https://avatars.discourse-cdn.com/v4/letter/n/73ab20/32.png) [@ntran](https://discuss.elastic.co/u/ntran)\ **Post date:** [January 15, 2020, 6:23am UTC](https://discuss.elastic.co/t/filebeat-cannot-find-registry-file-var-lib-filebeat-registry-filebeat/214997/3 "2020-01-15T06:23:54Z")
Hi @shaunak, Filebeat 7.2.0 I ran the command after starting Filebeat I've restart Filebeat again and I am still getting the same error message I ran filebeat -e and these are the logs from that. > 2020-01-15T06:06:31.454Z INFO instance/beat.go:292 Setup Beat: filebeat; Version: 7.2.0 > > 2020-01-15T06:06:31.454Z INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'filebeat-7.2.0' as ILM is enabled. > > 2020-01-15T06:06:31.454Z INFO elasticsearch/client.go:166 Elasticsearch url: [https://X:9200](https://X:9200) > > 2020-01-15T06:06:31.455Z INFO [publisher] pipeline/module.go:97 Beat name: elastic-nfs01 > > 2020-01-15T06:06:31.456Z INFO [monitoring] log/log.go:118 Starting metrics logging every 30s > > 2020-01-15T06:06:31.456Z INFO instance/beat.go:421 filebeat start running. > > 2020-01-15T06:06:31.456Z INFO registrar/registrar.go:145 Loading registrar data from /var/lib/filebeat/registry/filebeat/data.json > > 2020-01-15T06:06:31.457Z INFO registrar/registrar.go:152 States Loaded from registrar: 101 > > 2020-01-15T06:06:31.457Z INFO crawler/crawler.go:72 Loading Inputs: 2 > > 2020-01-15T06:06:31.462Z INFO log/input.go:148 Configured paths: > > 2020-01-15T06:06:31.462Z INFO input/input.go:114 Starting input of type: log; ID: X > > 2020-01-15T06:06:31.470Z INFO log/harvester.go:253 Harvester started for file: [file] > > 2020-01-15T06:06:31.492Z INFO input/input.go:114 Starting input of type: log; ID: X > > 2020-01-15T06:06:31.511Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:31.570Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:31.681Z INFO log/input.go:148 Configured paths: > > 2020-01-15T06:06:31.681Z INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 2 > > 2020-01-15T06:06:31.681Z INFO cfgfile/reload.go:172 Config reloader started > > 2020-01-15T06:06:31.702Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:31.837Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:31.877Z INFO elasticsearch/client.go:166 Elasticsearch url: [https://X:9200](https://X:9200) > > 2020-01-15T06:06:31.877Z WARN tlscommon/tls\_config.go:79 SSL/TLS verifications disabled. > > 2020-01-15T06:06:32.022Z INFO elasticsearch/client.go:735 Attempting to connect to Elasticsearch version 7.5.0 > > 2020-01-15T06:06:32.124Z INFO input/input.go:114 Starting input of type: log; ID: X > > 2020-01-15T06:06:32.125Z INFO cfgfile/reload.go:227 Loading of config files completed. > > 2020-01-15T06:06:34.569Z INFO elasticsearch/client.go:735 Attempting to connect to Elasticsearch version 7.5.0 > > 2020-01-15T06:06:34.729Z INFO [index-management] idxmgmt/std.go:252 Auto ILM enable success. > > 2020-01-15T06:06:34.740Z INFO [index-management.ilm] ilm/std.go:134 do not generate ilm policy: exists=true, overwrite=false > > 2020-01-15T06:06:34.740Z INFO [index-management] idxmgmt/std.go:265 ILM policy successfully loaded. > > 2020-01-15T06:06:34.740Z INFO [index-management] idxmgmt/std.go:394 Set setup.template.name to '{filebeat-7.2.0 {now/d}-000001}' as ILM is enabled. > > 2020-01-15T06:06:34.740Z INFO [index-management] idxmgmt/std.go:399 Set setup.template.pattern to 'filebeat-7.2.0-\*' as ILM is enabled. > > 2020-01-15T06:06:34.740Z INFO [index-management] idxmgmt/std.go:433 Set settings.index.lifecycle.rollover\_alias in template to {filebeat-7.2.0 {now/d}-000001} as ILM is enabled. > > 2020-01-15T06:06:34.740Z INFO [index-management] idxmgmt/std.go:437 Set settings.index.lifecycle.name in template to {filebeat-7.2.0 {"policy":{"phases":{"hot":{"actions":{"rollover":{"max\_age":"30d","max\_size":"50gb"}}}}}}} as ILM is enabled. > > 2020-01-15T06:06:34.745Z INFO template/load.go:88 Template filebeat-7.2.0 already exists and will not be overwritten. > > 2020-01-15T06:06:34.745Z INFO [index-management] idxmgmt/std.go:289 Loaded index template. > > 2020-01-15T06:06:34.747Z INFO [index-management] idxmgmt/std.go:300 Write alias successfully generated. > > 2020-01-15T06:06:34.857Z INFO pipeline/output.go:105 Connection to backoff(elasticsearch([https://10.61.13.13:9200](https://10.61.13.13:9200))) established > > 2020-01-15T06:06:34.864Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:35.622Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:35.886Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:36.248Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:37.362Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:37.514Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:38.772Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:39.331Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:39.686Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:40.618Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:41.236Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:41.471Z INFO log/harvester.go:253 Harvester started for file: /data/hrc-elastic-backup/logs/hrc-elastic-hotdata02/hrc-elastic-01\_server.json > > 2020-01-15T06:06:42.069Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:44.581Z INFO log/harvester.go:253 Harvester started for file: /data/kca-elastic-backup/logs/kca-elastic-hotdata01/kca-cluster-01\_audit.json > > 2020-01-15T06:06:44.939Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:45.381Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:46.348Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:49.002Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:50.904Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > 2020-01-15T06:06:51.222Z ERROR registrar/registrar.go:374 Writing of registry returned error: rename /var/lib/filebeat/registry/filebeat/data.json.new /var/lib/filebeat/registry/filebeat/data.json: no such file or directory. Continuing... > > ^C2020-01-15T06:06:51.472Z INFO beater/filebeat.go:433 Stopping filebeat > > 2020-01-15T06:06:51.472Z INFO crawler/crawler.go:139 Stopping Crawler > > 2020-01-15T06:06:51.472Z INFO crawler/crawler.go:149 Stopping 2 inputs > > 2020-01-15T06:06:51.472Z INFO cfgfile/reload.go:230 Dynamic config reloader stopped > > 2020-01-15T06:06:51.472Z INFO [reload] cfgfile/list.go:118 Stopping 1 runners ... > > 2020-01-15T06:06:51.472Z INFO input/input.go:149 input ticker stopped > > 2020-01-15T06:06:51.472Z INFO input/input.go:167 Stopping Input: 974067711281631264 > > 2020-01-15T06:06:51.472Z INFO input/input.go:149 input ticker stopped > > 2020-01-15T06:06:51.472Z INFO input/input.go:167 Stopping Input: 687122273758875045 ---
**Author:** ![ntran](https://avatars.discourse-cdn.com/v4/letter/n/73ab20/32.png) [@ntran](https://discuss.elastic.co/u/ntran)\ **Post date:** [January 15, 2020, 6:24am UTC](https://discuss.elastic.co/t/filebeat-cannot-find-registry-file-var-lib-filebeat-registry-filebeat/214997/4 "2020-01-15T06:24:48Z")
Filebeat is currently active (running) and I can see it is harvesting logs: > Jan 15 06:23:53 elastic-nfs01 filebeat[42359]: 2020-01-15T06:23:53.431Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":12240,"time":{"ms":304}},"total":{"ticks":155340,"time":{"ms":3596},"value":155340},"user":{"ticks":143100,"time":{"ms":3292}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":90},"info":{"ephemeral\_id":"X","uptime":{"ms":1050108}},"memstats":{"gc\_next":90491200,"memory\_alloc":78941936,"memory\_total":17877918144},"runtime":{"goroutines":406}},"filebeat":{"events":{"active":1,"added":15387,"done":15386},"harvester":{"open\_files":82,"running":82}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":15386,"batches":308,"total":15386},"read":{"bytes":3567224},"write":{"bytes":10595024}},"pipeline":{"clients":10,"events":{"active":4117,"published":15386,"total":15386},"queue":{"acked":15386}}},"registrar":{"states":{"current":101,"update":15386},"writes":{"success":232,"total":232}},"system":{"load":{"1":0.03,"15":0.41,"5":0.19,"norm":{"1":0.0075,"15":0.1025,"5":0.0475}}}}}} When I also look in `/var/lib/filebeat/registry/filebeat/data.json`, the timestamp is today, denoting the file is being harvested by Filebeat. So I'm really confused right now in regards to what is happening. > {"source":"/data/elastic-backup/logs/elastic-warmdata03/elastic-01\_audit-2019-12-15.log","offset":18049495555,"timestamp":"2020-01-15T06:29:09.344601377Z","ttl":-1,"type":"log","meta":null,"FileStateOS":{"inode":238781751,"device":64769}}] ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [January 15, 2020, 4:09pm UTC](https://discuss.elastic.co/t/filebeat-cannot-find-registry-file-var-lib-filebeat-registry-filebeat/214997/5 "2020-01-15T16:09:30Z")
Are you, by any chance, running multiple Filebeat instances on the same host, which could be sharing the same registry path? ---
**Author:** ![ntran](https://avatars.discourse-cdn.com/v4/letter/n/73ab20/32.png) [@ntran](https://discuss.elastic.co/u/ntran)\ **Post date:** [January 16, 2020, 6:36am UTC](https://discuss.elastic.co/t/filebeat-cannot-find-registry-file-var-lib-filebeat-registry-filebeat/214997/6 "2020-01-16T06:36:02Z")
When I run `filebeat export config`, I get the following result: As far as I am aware, I am only running one filebeat instance > filebeat: > config: > modules: > path: /etc/filebeat/modules.d/\*.yml > reload: > enabled: false > inputs: > > - clean\_inactive: 72h > clean\_removed: true > filebeat: > registry: > path: /var/lib/filebeat/registry > ignore\_older: 48h > paths: > - /data/name-elastic-backup/logs/name-elastic-\*/name-elastic-01\_server.json > type: log > > - clean\_inactive: 72h > clean\_removed: true > filebeat: > registry: > path: /var/lib/filebeat/registry > ignore\_older: 48h > paths: > - /data/elastic-backup/logs/elastic\*/cluster-01\_server.json > type: log > output: > elasticsearch: > bulk\_max\_size: 50 > compression\_level: 1 > hosts: > - XX.XX.XX.XX:9200 > password: changeme > protocol: https > ssl: > verification\_mode: none > username: elastic > worker: 2 > path: > config: /etc/filebeat > data: /var/lib/filebeat > home: /usr/share/filebeat > logs: /var/log/filebeat > processors: > > - add\_host\_metadata: null > - add\_cloud\_metadata: null > setup: > kibana: > host: [https://XX.XX.XX.XX:5601](https://XX.XX.XX.XX:5601) > ssl: > verification\_mode: none > template: > settings: > index: > number\_of\_shards: 1 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 13, 2020, 6:36am UTC](https://discuss.elastic.co/t/filebeat-cannot-find-registry-file-var-lib-filebeat-registry-filebeat/214997/7 "2020-02-13T06:36:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.