Hi,
I am using Filebeat version 6.6.2 on macOS. The configuration files is:
filebeat.prospectors:
- type: log
enabled: true
paths:
- /tmp/*.json
json.keys_under_root: true
json.overwrite_keys: true
output.logstash:
enabled: true
hosts: ["localhost:32771"]
index: "nslogs"
setup.template.name: "nslogs"
setup.template.pattern: "nslogs"
I am running it with this command filebeat --path.config . -c filebeat_nslogs.yml -e -v run.
What I see is that the files are harvested correctly, but then nothing happens with them even though I specified the output to logstash. I used to see some logs when filebeat was trying to connect and send the files, but now I don't receive anything. I don't know what has changed. I've even tried to put a random port to see if I receive Connection refused and I've tried to listen with a process on 32771 to see if something is received. Nothing worked.
Filebeat logs:
BUH-M-W21M5H:log-tools bvlad$ ./filebeat_nslogs_run.sh
2019-03-19T18:51:08.396+0200 INFO instance/beat.go:616 Home path: [/Users/bvlad/Documents/log-tools] Config path: [.] Data path: [/Users/bvlad/Documents/log-tools/data] Logs path: [/Users/bvlad/Documents/log-tools/logs]
2019-03-19T18:51:08.396+0200 INFO instance/beat.go:623 Beat UUID: 813e4987-7c5a-47aa-b782-bf7d950525bd
2019-03-19T18:51:08.396+0200 INFO [beat] instance/beat.go:936 Beat info {"system_info": {"beat": {"path": {"config": ".", "data": "/Users/bvlad/Documents/log-tools/data", "home": "/Users/bvlad/Documents/log-tools", "logs": "/Users/bvlad/Documents/log-tools/logs"}, "type": "filebeat", "uuid": "813e4987-7c5a-47aa-b782-bf7d950525bd"}}}
2019-03-19T18:51:08.396+0200 INFO [beat] instance/beat.go:945 Build info {"system_info": {"build": {"commit": "1eea934ce81be553337f2828bd12131896fea8e4", "libbeat": "6.6.2", "time": "2019-03-06T14:17:56.000Z", "version": "6.6.2"}}}
2019-03-19T18:51:08.396+0200 INFO [beat] instance/beat.go:948 Go runtime info {"system_info": {"go": {"os":"darwin","arch":"amd64","max_procs":8,"version":"go1.10.8"}}}
2019-03-19T18:51:08.398+0200 INFO [beat] instance/beat.go:952 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-03-19T11:39:48.660153+02:00","name":"BUH-M-W21M5H","ip":["127.0.0.1/8","::1/128","fe80::1/64","fe80::1c62:a09d:3844:b169/64","192.168.100.14/24","2a02:2f0a:c301:9500:1c82:95b2:f291:7806/64","2a02:2f0a:c301:9500:648c:be76:af56:91f6/64","fe80::6834:bdff:fee6:6ee/64","fe80::5c8d:30d5:d4e3:a69e/64","fe80::ad68:2413:708f:e48e/64","fe80::b6ec:d29a:2771:77bd/64","fe80::aede:48ff:fe00:1122/64"],"kernel_version":"17.7.0","mac":["8c:85:90:a4:fa:37","0e:85:90:a4:fa:37","6a:34:bd:e6:06:ee","fe:00:68:22:f3:01","fe:00:68:22:f3:00","fe:00:68:22:f3:05","fe:00:68:22:f3:04","fe:00:68:22:f3:01","ac:de:48:00:11:22"],"os":{"family":"darwin","platform":"darwin","name":"Mac OS X","version":"10.13.6","major":10,"minor":13,"patch":6,"build":"17G65"},"timezone":"EET","timezone_offset_sec":7200,"id":"BA66E02C-2736-5CD7-8ECB-09A9EFDACBE2"}}}
2019-03-19T18:51:08.399+0200 INFO [beat] instance/beat.go:981 Process info {"system_info": {"process": {"cwd": "/Users/bvlad/Documents/log-tools", "exe": "/Users/bvlad/filebeat-6.6.2-darwin-x86_64/filebeat", "name": "filebeat", "pid": 18010, "ppid": 18009, "start_time": "2019-03-19T18:51:08.352+0200"}}}
2019-03-19T18:51:08.399+0200 INFO instance/beat.go:281 Setup Beat: filebeat; Version: 6.6.2
2019-03-19T18:51:08.400+0200 INFO [publisher] pipeline/module.go:110 Beat name: BUH-M-W21M5H
2019-03-19T18:51:08.400+0200 WARN [cfgwarn] beater/filebeat.go:81 DEPRECATED: prospectors are deprecated, Use `inputs` instead. Will be removed in version: 7.0.0
2019-03-19T18:51:08.401+0200 ERROR fileset/modules.go:118 Not loading modules. Module directory not found: /Users/bvlad/Documents/log-tools/module
2019-03-19T18:51:08.401+0200 INFO instance/beat.go:403 filebeat start running.
2019-03-19T18:51:08.401+0200 INFO [monitoring] log/log.go:117 Starting metrics logging every 30s
2019-03-19T18:51:08.402+0200 INFO registrar/registrar.go:134 Loading registrar data from /Users/bvlad/Documents/log-tools/data/registry
2019-03-19T18:51:08.402+0200 INFO registrar/registrar.go:141 States Loaded from registrar: 3
2019-03-19T18:51:08.402+0200 WARN beater/filebeat.go:367 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2019-03-19T18:51:08.402+0200 INFO crawler/crawler.go:72 Loading Inputs: 1
2019-03-19T18:51:08.404+0200 INFO log/input.go:138 Configured paths: [/tmp/*.json]
2019-03-19T18:51:08.404+0200 INFO input/input.go:114 Starting input of type: log; ID: 2525578105644193438
2019-03-19T18:51:08.404+0200 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2019-03-19T18:51:28.410+0200 INFO log/harvester.go:255 Harvester started for file: /tmp/processed.somefilenameee1.json
2019-03-19T18:51:28.411+0200 INFO log/harvester.go:255 Harvester started for file: /tmp/processed.somefilenameee2.json
2019-03-19T18:51:28.411+0200 INFO log/harvester.go:255 Harvester started for file: /tmp/processed.somefilenameee3.json
2019-03-19T18:51:38.408+0200 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":34,"time":{"ms":34}},"total":{"ticks":66,"time":{"ms":66},"value":66},"user":{"ticks":32,"time":{"ms":32}}},"info":{"ephemeral_id":"44a04934-7843-416f-85fe-704fa80f6ce1","uptime":{"ms":30022}},"memstats":{"gc_next":4194304,"memory_alloc":1832480,"memory_total":4673728,"rss":19546112}},"filebeat":{"events":{"added":9,"done":9},"harvester":{"open_files":3,"running":3,"started":3}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"logstash"},"pipeline":{"clients":1,"events":{"active":0,"filtered":9,"total":9}}},"registrar":{"states":{"cleanup":3,"current":3,"update":9},"writes":{"success":9,"total":9}},"system":{"cpu":{"cores":8},"load":{"1":2.5596,"15":8.1797,"5":3.2202,"norm":{"1":0.3199,"15":1.0225,"5":0.4025}}}}}}
I have the same problem when I host ELK stack and Filebeat with Docker. The example that I've gave you above is just Filebeat process started manually. I've tried to isolate the problem away from the Docker infrastructure.
Thanks,
Bogdan