Hi, Im am testing configurations for a log file, and I want to use it many times, until the desired result is achieved, the problem is that I can not find the data/registry folder where the already processed files are checked. to delete it and try the same log file again
I have search the folowing paths, but they not exist:
@Incauto - As you can read in our documentation, the registry path will be, by default, set to ${path.data}/registry. Depending on your installation "type" (i.e RPM, tarball, etc.), path.data will correspond to a different value/location as described here.
Take note that, the registry stores its data in the subdirectory filebeat/data.json. It also contains a meta data file named filebeat/meta.json. The meta file contains the file format version number.
Tanks ropc, I have read the documentation, I have instaled filebeat via rpm, but there is no
/var/lib/filebeat folder, no changes on the filebeat.yml only in the input :
@Incauto - I just tested in the lab with an RPM installation. The registry folder should be in /var/lib/filebeat/registry. If you are running Filebeat with sudo service filebeat start, it would be good to check the logs from journald. For example, you could run:
sudo journalctl -u filebeat > export.log and check the logs. You should find similar logs that will indicate where the registry / data files are located:
INFO instance/beat.go:610 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
....
INFO registrar/migrate.go:104 No registry home found. Create: /var/lib/filebeat/registry/filebeat
INFO registrar/migrate.go:112 Initialize registry meta file
INFO registrar/registrar.go:108 No registry file found under: /var/lib/filebeat/registry/filebeat/data.json. Creating a new registry file.
INFO registrar/registrar.go:145 Loading registrar data from /var/lib/filebeat/registry/filebeat/data.json
INFO registrar/registrar.go:152 States Loaded from registrar: 0
...
Alternatively, you can also run Filebeat in the foreground with debug logging with the following command:
sudo filebeat -d "*" -e , which will give you similar logs with additional debug logs. That could be handy, especially if there is a problem with Filebeat or the configuration.
so, I was running filebeat with this command, similar when I run a conf in logstash:
bin/filebeat -c /etc/filebeat/filebeat.yml
I never start filebeat as a service, maybe thats why the directories weren't created, still a mystery where the pointer where stored before, but its all working now.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.