I have configured elk stack with filebeat.
I am standing logs from filebeat to logstash but it's not running.
The configuration in filebeat.yml :
filebeat.inputs:
- type: log
enabled: true
-C:/Users/mohnadik/Downloads/abc.log
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
setup.template.settings:
index.number_of_shards: 1
#Logstash
output.logstash:
hosts: ["localhost:5044"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
The conf file in logstash is :
input{
file{
path => "C:/Users/mohnadik/Downloads/abc.log"
start_position => "beginning"
sincedb_path => "NUL"
}}
output{stdout{} }
The log files from filebeat is :
2020-06-30T13:57:34.342+0530 INFO instance/beat.go:622 Home path: [C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64] Config path: [C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64] Data path: [C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64\data] Logs path: [C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64\logs]
2020-06-30T13:57:34.344+0530 DEBUG [beat] instance/beat.go:674 Beat metadata path: C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64\data\meta.json
2020-06-30T13:57:34.350+0530 INFO instance/beat.go:630 Beat ID: c73ce035-bac3-49f3-90b5-05cee727c8c9
2020-06-30T13:57:34.403+0530 DEBUG [filters] add_cloud_metadata/providers.go:126 add_cloud_metadata: starting to fetch metadata, timeout=3s
2020-06-30T13:57:34.433+0530 INFO [beat] instance/beat.go:958 Beat info {"system_info": {"beat": {"path": {"config": "C:\\Users\\mohnadik\\Downloads\\filebeat-7.6.2-windows-x86_64", "data": "C:\\Users\\mohnadik\\Downloads\\filebeat-7.6.2-windows-x86_64\\data", "home": "C:\\Users\\mohnadik\\Downloads\\filebeat-7.6.2-windows-x86_64", "logs": "C:\\Users\\mohnadik\\Downloads\\filebeat-7.6.2-windows-x86_64\\logs"}, "type": "filebeat", "uuid": "c73ce035-bac3-49f3-90b5-05cee727c8c9"}}}
2020-06-30T13:57:34.434+0530 INFO [beat] instance/beat.go:967 Build info {"system_info": {"build": {"commit": "d57bcf8684602e15000d65b75afcd110e2b12b59", "libbeat": "7.6.2", "time": "2020-03-26T05:23:36.000Z", "version": "7.6.2"}}}
2020-06-30T13:57:34.434+0530 INFO [beat] instance/beat.go:970 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.13.8"}}}
2020-06-30T13:57:34.485+0530 INFO [beat] instance/beat.go:974 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-06-30T11:23:45.82+05:30","name":"WL310610","ip":["fe80::51b7:4fe5:3e53:1585/64","169.254.21.133/16","fe80::8486:fde8:8dec:f973/64","169.254.249.115/16","fe80::d120:573:14d6:2762/64","169.254.39.98/16","fe80::4922:27b6:9d4a:3bce/64","192.168.79.1/24","fe80::8148:597b:be37:b127/64","192.168.23.1/24","fe80::6c79:be4:973b:84f1/64","192.168.1.3/24","::1/128","127.0.0.1/8"],"kernel_version":"10.0.17763.1282 (WinBuild.160101.0800)","mac":["20:47:47:c0:bf:ff","48:45:20:9f:51:4c","4a:45:20:9f:51:4b","00:50:56:c0:00:01","00:50:56:c0:00:08","48:45:20:9f:51:4b"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Enterprise","version":"10.0","major":10,"minor":0,"patch":0,"build":"17763.1282"},"timezone":"IST","timezone_offset_sec":19800,"id":"4a33b7c7-869c-47b3-a476-50c7caabdfc2"}}}
2020-06-30T13:57:34.498+0530 INFO [beat] instance/beat.go:1003 Process info {"system_info": {"process": {"cwd": "C:\\Users\\mohnadik\\Downloads\\filebeat-7.6.2-windows-x86_64", "exe": "C:\\Users\\mohnadik\\Downloads\\filebeat-7.6.2-windows-x86_64\\filebeat.exe", "name": "filebeat.exe", "pid": 9976, "ppid": 16448, "start_time": "2020-06-30T13:57:31.643+0530"}}}
2020-06-30T13:57:34.498+0530 INFO instance/beat.go:298 Setup Beat: filebeat; Version: 7.6.2
2020-06-30T13:57:34.500+0530 INFO [publisher] pipeline/module.go:110 Beat name: WL310610
2020-06-30T13:57:34.504+0530 WARN beater/filebeat.go:152 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-06-30T13:57:34.505+0530 INFO instance/beat.go:439 filebeat start running.
2020-06-30T13:57:34.505+0530 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2020-06-30T13:57:34.505+0530 DEBUG [service] service/service_windows.go:72 Windows is interactive: true
2020-06-30T13:57:34.506+0530 DEBUG [test] registrar/migrate.go:159 isFile(C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64\data\registry) -> false
2020-06-30T13:57:34.514+0530 DEBUG [test] registrar/migrate.go:159 isFile() -> false
2020-06-30T13:57:34.515+0530 DEBUG [test] registrar/migrate.go:152 isDir(C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64\data\registry\filebeat) -> true
2020-06-30T13:57:34.515+0530 DEBUG [test] registrar/migrate.go:159 isFile(C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64\data\registry\filebeat\meta.json) -> true
2020-06-30T13:57:34.516+0530 DEBUG [registrar] registrar/migrate.go:51 Registry type '0' found
2020-06-30T13:57:34.517+0530 DEBUG [registrar] registrar/registrar.go:125 Registry file set to: C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64\data\registry\filebeat\data.json
2020-06-30T13:57:34.525+0530 INFO registrar/registrar.go:145 Loading registrar data from C:\Users\mohnadik\Downloads\filebeat-7.6.2-windows-x86_64\data\registry\filebeat\data.json
2020-06-30T13:57:34.526+0530 INFO registrar/registrar.go:152 States Loaded from registrar: 0
Can some please suggest me what I am doing here?
Any help would be really appreciated.