when start filebeat （1.2.3） whith the path which have 270000 files but the size only 3G,it just harvester no publish.
the dug log:
below is the config:
filebeat: paths: prospectors: - /data1/logs/**/*.txt input_type: log close_older: 1m document_type: xzb_logs scan_frequency: 30s backoff: 1h max_backoff: 24h spool_size: 16048 idle_timeout: 5s registry_file: /usr/local/filebeat/filebeat-1.2.3-x86_64/filebeat_registry output: logstash: hosts: ["10.162.10.34:5044"] worker: 2 logging: to_files: true path: /usr/local/filebeat/logs name: mybeat.log rotateeverybytes: 10485760 keepfiles: 10 level: debug
when i change the path with less files ,it works well.
my logs files uploaded by other system and do not update again
oh no,it start publishing after about 1 h. but i config the idle_timeout with 5s, it should pulish per 5s, i cant understant it .