Hello!
Can anyone confirm that this is the expected output of filebeats when it successfully harvests and ships the associated data to logstash?
filebeats.yml:
filebeat.inputs:
- type: log
paths:
- c:\elk\logs\test.log
output.logstash:
hosts: ["localhost:5044"]
logstash conf:
input {
beats {
port => "5044"
}
}
output {
stdout { codec => rubydebug }
}
filebeats log output:
2019-08-22T06:52:15.307-0700 INFO instance/beat.go:606 Home path: [C:\elk\filebeat] Config path: [C:\elk\filebeat] Data path: [C:\elk\filebeat\data] Logs path: [C:\elk\filebeat\logs]
2019-08-22T06:52:15.309-0700 INFO instance/beat.go:614 Beat ID: ac845bac-bb74-4517-8732-80c6ef802e05
2019-08-22T06:52:15.309-0700 INFO [beat] instance/beat.go:902 Beat info {"system_info": {"beat": {"path": {"config": "C:\\elk\\filebeat", "data": "C:\\elk\\filebeat\\data", "home": "C:\\elk\\filebeat", "logs": "C:\\elk\\filebeat\\logs"}, "type": "filebeat", "uuid": "ac845bac-bb74-4517-8732-80c6ef802e05"}}}
2019-08-22T06:52:15.309-0700 INFO [beat] instance/beat.go:911 Build info {"system_info": {"build": {"commit": "6f0ec01a0e57fe7d4fd703b017fb5a2f6448d097", "libbeat": "7.3.0", "time": "2019-07-24T17:39:33.000Z", "version": "7.3.0"}}}
2019-08-22T06:52:15.309-0700 INFO [beat] instance/beat.go:914 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.12.4"}}}
2019-08-22T06:52:15.314-0700 INFO [beat] instance/beat.go:918 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-08-21T20:52:23.4-07:00","name":"vagrant-2012-r2","ip":["fe80::4c67:7188:66c1:e795/64","10.0.2.15/24","::1/128","127.0.0.1/8","2001:0:5cf2:8c02:1c54:153c:f5ff:fdf0/64","fe80::1c54:153c:f5ff:fdf0/64","fe80::5efe:a00:20f/128"],"kernel_version":"6.3.9600.19426 (winblue_ltsb_escrow.190715-1750)","mac":["08:00:27:e9:96:13","00:00:00:00:00:00:00:e0","00:00:00:00:00:00:00:e0"],"os":{"family":"windows","platform":"windows","name":"Windows Server 2012 R2 Standard","version":"6.3","major":3,"minor":0,"patch":0,"build":"9600.19431"},"timezone":"PDT","timezone_offset_sec":-25200,"id":"42f11c3b-3062-4874-9ea3-50e0e08c4434"}}}
2019-08-22T06:52:15.315-0700 INFO [beat] instance/beat.go:947 Process info {"system_info": {"process": {"cwd": "C:\\elk\\filebeat", "exe": "C:\\elk\\filebeat\\filebeat.exe", "name": "filebeat.exe", "pid": 3560, "ppid": 2960, "start_time": "2019-08-22T06:52:15.268-0700"}}}
2019-08-22T06:52:15.315-0700 INFO instance/beat.go:292 Setup Beat: filebeat; Version: 7.3.0
2019-08-22T06:52:15.315-0700 INFO [publisher] pipeline/module.go:97 Beat name: vagrant-2012-r2
2019-08-22T06:52:15.316-0700 WARN beater/filebeat.go:152 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2019-08-22T06:52:15.316-0700 INFO instance/beat.go:421 filebeat start running.
2019-08-22T06:52:15.316-0700 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2019-08-22T06:52:15.317-0700 INFO registrar/registrar.go:145 Loading registrar data from C:\elk\filebeat\data\registry\filebeat\data.json
2019-08-22T06:52:15.317-0700 INFO registrar/registrar.go:152 States Loaded from registrar: 1
2019-08-22T06:52:15.317-0700 WARN beater/filebeat.go:368 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2019-08-22T06:52:15.317-0700 INFO crawler/crawler.go:72 Loading Inputs: 1
2019-08-22T06:52:15.317-0700 INFO log/input.go:148 Configured paths: [c:\elk\logs\test.log]
2019-08-22T06:52:15.317-0700 INFO input/input.go:114 Starting input of type: log; ID: 2903726485811444168
2019-08-22T06:52:15.317-0700 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2019-08-22T06:52:15.317-0700 INFO log/harvester.go:253 Harvester started for file: c:\elk\logs\test.log
2019-08-22T06:52:46.155-0700 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":15,"time":{"ms":15}},"total":{"ticks":30,"time":{"ms":30},"value":0},"user":{"ticks":15,"time":{"ms":15}}},"handles":{"open":141},"info":{"ephemeral_id":"b4ca2c98-9110-441c-adfa-d1c03b8bc488","uptime":{"ms":30045}},"memstats":{"gc_next":7761456,"memory_alloc":4465120,"memory_total":7445408,"rss":22880256},"runtime":{"goroutines":26}},"filebeat":{"events":{"added":2,"done":2},"harvester":{"open_files":1,"running":1,"started":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"logstash"},"pipeline":{"clients":1,"events":{"active":0,"filtered":2,"total":2}}},"registrar":{"states":{"current":1,"update":2},"writes":{"success":2,"total":2}},"system":{"cpu":{"cores":4}}}}}
logstash result:
(not receiving anything - seemingly)