I don't see any changes in the log for a long time
2019-10-29T07:25:38.891Z INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":2500,"time":2503},"total":{"ticks":8140,"time":8145,"value":8140},"user":{"ticks":5640,"time":5642}},"info":{"ephemeral_id":"e0bd341a-6d22-4d36-937a-ec477851f9f3","uptime":{"ms":24780010}},"memstats":{"gc_next":4881376,"memory_alloc":4255320,"memory_total":1641005672}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":459}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0.3,"15":0.03,"5":0.13,"norm":{"1":0.15,"15":0.015,"5":0.065}}}}}}
2019-10-29T07:26:08.891Z INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":2510,"time":2510},"total":{"ticks":8150,"time":8157,"value":8150},"user":{"ticks":5640,"time":5647}},"info":{"ephemeral_id":"e0bd341a-6d22-4d36-937a-ec477851f9f3","uptime":{"ms":24810010}},"memstats":{"gc_next":5866896,"memory_alloc":4138808,"memory_total":1643016232}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":459}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0.26,"15":0.04,"5":0.14,"norm":{"1":0.13,"15":0.02,"5":0.07}}}}}}
-----
2019-10-29T07:35:38.891Z INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":2560,"time":2565},"total":{"ticks":8340,"time":8346,"value":8340},"user":{"ticks":5780,"time":5781}},"info":{"ephemeral_id":"e0bd341a-6d22-4d36-937a-ec477851f9f3","uptime":{"ms":25380010}},"memstats":{"gc_next":5852736,"memory_alloc":3530312,"memory_total":1681225912}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":459}}},"registrar":{"states":{"current":1}},"system":{"load":{"1":0.01,"15":0.01,"5":0.04,"norm":{"1":0.005,"15":0.005,"5":0.02}}}}}}
But I have the log files I need to collect
/var/www/webapp/logs/A/B-FILES:
total 60K
-rw-r--r-- 1 root root 50K Oct 29 00:34 B-FILES.log
My configuration file is
sh-4.2$ sudo cat /etc/filebeat/filebeat.yml
filebeat.prospectors:
- input_type: log
paths:
- /var/applog/**/*.log
- /var/www/webapp/logs/**/*.log
multiline:
pattern: '^\d+-\d+-\d+ \d+:\d+:\d+\.\d+|^\d+\.\d+\.\d+\.\d+|^\[\d+-\d+-\d+ \d+:\d+:\d+\]|^\d+:\d+:\d+.\d+|^\d+-\d+-\d+T\d+:\d+:\d+.\d+\+\d+:\d+|^\d{4}\\\d{2}\\\d{2} \d{2}:\d{2}:\d{2} \[|^\d+-\d+-\d+ \d+:\d+:\d+|^\d+:\d+:\d+|^\[\d+-[[:alpha:]]+-\d+ \d+:\d+:\d+\] '
negate: true
match: after
But when I execute debug, it has a collection log
sh-4.2$ sudo filebeat -e -d "*"
2019-10-29T07:38:10.018Z INFO instance/beat.go:468 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2019-10-29T07:38:10.018Z DEBUG [beat] instance/beat.go:495 Beat metadata path: /var/lib/filebeat/meta.json
2019-10-29T07:38:10.019Z INFO instance/beat.go:475 Beat UUID: 7a4c3a40-9940-46f4-a9f7-33552ad11ae5
2019-10-29T07:38:10.019Z INFO instance/beat.go:213 Setup Beat: filebeat; Version: 6.2.1
2019-10-29T07:38:10.019Z DEBUG [beat] instance/beat.go:230 Initializing output plugins
2019-10-29T07:38:10.019Z DEBUG [processors] processors/processor.go:49 Processors:
2019-10-29T07:38:10.019Z INFO pipeline/module.go:76 Beat name: ip-10-10-12-206
2019-10-29T07:38:10.019Z INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2019-10-29T07:38:10.019Z INFO instance/beat.go:301 filebeat start running.
2019-10-29T07:38:10.020Z DEBUG [registrar] registrar/registrar.go:88 Registry file set to: /var/lib/filebeat/registry
2019-10-29T07:38:10.020Z INFO registrar/registrar.go:108 Loading registrar data from /var/lib/filebeat/registry
2019-10-29T07:38:10.020Z INFO registrar/registrar.go:119 States Loaded from registrar: 6
2019-10-29T07:38:10.020Z WARN beater/filebeat.go:261 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If youhave already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2019-10-29T07:38:10.020Z INFO crawler/crawler.go:48 Loading Prospectors: 1
2019-10-29T07:38:10.020Z WARN [cfgwarn] prospector/config.go:25 DEPRECATED: input_type prospector config is deprecated. Use type instead. Will be removed in version: 6.0.0
2019-10-29T07:38:10.020Z DEBUG [processors] processors/processor.go:49 Processors:
2019-10-29T07:38:10.020Z DEBUG [registrar] registrar/registrar.go:150 Starting Registrar
2019-10-29T07:38:10.020Z DEBUG [prospector] log/config.go:178 recursive glob enabled
2019-10-29T07:38:10.020Z DEBUG [prospector] log/config.go:186 "/var/applog/**/*.log" expanded to []string{"/var/applog/*.log", "/var/applog/*/*.log", "/var/applog/*/*/*.log", "/var/applog/*/*/*/*.log", "/var/applog/*/*/*/*/*.log", "/var/applog/*/*/*/*/*/*.log", "/var/applog/*/*/*/*/*/*/*.log", "/var/applog/*/*/*/*/*/*/*/*.log", "/var/applog/*/*/*/*/*/*/*/*/*.log"}
2019-10-29T07:38:10.020Z DEBUG [prospector] log/config.go:186 "/var/www/webapp/logs/**/*.log" expanded to []string{"/var/www/webapp/logs/*.log", "/var/www/webapp/logs/*/*.log", "/var/www/webapp/logs/*/*/*.log", "/var/www/webapp/logs/*/*/*/*.log", "/var/www/webapp/logs/*/*/*/*/*.log", "/var/www/webapp/logs/*/*/*/*/*/*.log", "/var/www/webapp/logs/*/*/*/*/*/*/*.log", "/var/www/webapp/logs/*/*/*/*/*/*/*/*.log", "/var/www/webapp/logs/*/*/*/*/*/*/*/*/*.log"}
2019-10-29T07:38:10.021Z DEBUG [prospector] log/prospector.go:120 exclude_files: []. Number of stats: 6
2019-10-29T07:38:10.021Z DEBUG [prospector] file/state.go:82 New state added for /var/www/webapp/logs/A/B-FILES/B-FILES.log
Why does debug start collecting when it doesn't work for a long time， Debug or restart and it will work properly