Hi all,
I am newbie to elk and filebeat. I have deployed EK(version 7.6.1) operator and logstash(version 7.6.1) in openshift. This scenaio works fine when I upload logs file to logstash. Now I am want to use filebeat to send data to logstash. I have Filebeat running as a docker container on Mac. On startup of filebeat docker I was expecting to see some logging with regards to connection success/failure to logstash. Also on startup I am mounting local log files to /usr/share/filebeaat/data folder but I don't see any processing logs. Any advice, much appreciated.
Thanks
Raj
Filebeat.yml
ilebeat.config:
modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
output.logstash:
hosts: '${LOGSTASH_HOSTS}'
timeout: 15
ssl:
certificate_authorities:
- /etc/pki/tls/certs/logstash-beats.crt
inputs:
- type: log
paths:
- /usr/share/filebeat/data/*.xml
filebeat.modules:
- module: logstash
logging.level: DEBUG
Start filebeat
$ docker run -d --rm --name=filebeat --user=root --volume="$(pwd)/development/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" --volume="$(pwd)/development/tls.crt:/etc/pki/tls/certs/logstash-beats.crt:ro" --volume="$(pwd)/development/facetlogs:/usr/share/filebeat/data" docker.elastic.co/beats/filebeat:7.6.1 filebeat -e -strict.perms=false -E LOGSTASH_HOSTS=["logstash-logging.ip.dev.com"]
Filebeat startup logs
2020-03-20T14:45:00.653Z INFO instance/beat.go:622 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2020-03-20T14:45:00.653Z DEBUG [beat] instance/beat.go:674 Beat metadata path: /usr/share/filebeat/data/meta.json
2020-03-20T14:45:00.656Z INFO instance/beat.go:630 Beat ID: 055a89a5-9bb7-41fe-bb3f-d6869d2d9be0
2020-03-20T14:45:00.658Z INFO [beat] instance/beat.go:958 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "055a89a5-9bb7-41fe-bb3f-d6869d2d9be0"}}}
2020-03-20T14:45:00.658Z INFO [beat] instance/beat.go:967 Build info {"system_info": {"build": {"commit": "c1c49432bdc53563e63e9d684ca3e9843626e448", "libbeat": "7.6.1", "time": "2020-02-28T23:12:26.000Z", "version": "7.6.1"}}}
2020-03-20T14:45:00.659Z INFO instance/beat.go:298 Setup Beat: filebeat; Version: 7.6.1
2020-03-20T14:45:00.659Z DEBUG [beat] instance/beat.go:324 Initializing output plugins
2020-03-20T14:45:00.662Z DEBUG [tls] tlscommon/tls.go:155 successfully loaded CA certificate: /etc/pki/tls/certs/logstash-beats.crt
2020-03-20T14:45:00.662Z DEBUG [publisher] pipeline/consumer.go:137 start pipeline event consumer
2020-03-20T14:45:00.662Z INFO [publisher] pipeline/module.go:110 Beat name: 2d5e05bd6726
2020-03-20T14:45:00.664Z INFO beater/filebeat.go:92 Enabled modules/filesets: logstash (log, slowlog), ()
2020-03-20T14:45:00.664Z WARN beater/filebeat.go:152 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-03-20T14:45:00.665Z INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2020-03-20T14:45:00.665Z INFO instance/beat.go:439 filebeat start running.
2020-03-20T14:45:00.665Z WARN beater/filebeat.go:181 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-03-20T14:45:00.666Z DEBUG [test] registrar/migrate.go:159 isFile(/usr/share/filebeat/data/registry) -> false
2020-03-20T14:45:00.666Z DEBUG [test] registrar/migrate.go:159 isFile() -> false
2020-03-20T14:45:00.667Z DEBUG [test] registrar/migrate.go:152 isDir(/usr/share/filebeat/data/registry/filebeat) -> true
2020-03-20T14:45:00.668Z DEBUG [test] registrar/migrate.go:159 isFile(/usr/share/filebeat/data/registry/filebeat/meta.json) -> true
2020-03-20T14:45:00.670Z DEBUG [registrar] registrar/migrate.go:51 Registry type '0' found
2020-03-20T14:45:00.671Z DEBUG [registrar] registrar/registrar.go:125 Registry file set to: /usr/share/filebeat/data/registry/filebeat/data.json
2020-03-20T14:45:00.672Z INFO registrar/registrar.go:145 Loading registrar data from /usr/share/filebeat/data/registry/filebeat/data.json
2020-03-20T14:45:00.673Z INFO registrar/registrar.go:152 States Loaded from registrar: 0
2020-03-20T14:45:00.673Z WARN beater/filebeat.go:368 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-03-20T14:45:00.673Z INFO crawler/crawler.go:72 Loading Inputs: 2
2020-03-20T14:45:00.674Z DEBUG [registrar] registrar/registrar.go:278 Starting Registrar
2020-03-20T14:45:00.674Z DEBUG [conditions] conditions/conditions.go:98 New condition regexp: map[]
2020-03-20T14:45:00.674Z DEBUG [conditions] conditions/conditions.go:98 New condition !regexp: map[]
2020-03-20T14:45:00.674Z DEBUG [processors] processors/processor.go:101 Generated new processors: add_locale=[format=offset], condition=!regexp: map[]
2020-03-20T14:45:00.674Z DEBUG [input] log/config.go:204 recursive glob enabled
2020-03-20T14:45:00.674Z DEBUG [input] log/input.go:164 exclude_files: [(?-s:.)gz(?-m:$)]. Number of stats: 0
2020-03-20T14:45:00.675Z DEBUG [input] log/input.go:185 input with previous states loaded: 0
2020-03-20T14:45:00.675Z INFO log/input.go:152 Configured paths: [/var/log/logstash/logstash-plain*.log]
2020-03-20T14:45:00.675Z INFO input/input.go:114 Starting input of type: log; ID: 8601861250991668688
2020-03-20T14:45:00.675Z DEBUG [input] log/input.go:191 Start next scan
2020-03-20T14:45:00.675Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-03-20T14:45:00.675Z DEBUG [conditions] conditions/conditions.go:98 New condition regexp: map[]
2020-03-20T14:45:00.674Z DEBUG [conditions] conditions/conditions.go:98 New condition !regexp: map[]
2020-03-20T14:45:00.674Z DEBUG [processors] processors/processor.go:101 Generated new processors: add_locale=[format=offset], condition=!regexp: map[]
2020-03-20T14:45:00.674Z DEBUG [input] log/config.go:204 recursive glob enabled
2020-03-20T14:45:00.674Z DEBUG [input] log/input.go:164 exclude_files: [(?-s:.)gz(?-m:$)]. Number of stats: 0
2020-03-20T14:45:00.675Z DEBUG [input] log/input.go:185 input with previous states loaded: 0
2020-03-20T14:45:00.675Z INFO log/input.go:152 Configured paths: [/var/log/logstash/logstash-plain*.log]
2020-03-20T14:45:00.675Z INFO input/input.go:114 Starting input of type: log; ID: 8601861250991668688
2020-03-20T14:45:00.675Z DEBUG [input] log/input.go:191 Start next scan
2020-03-20T14:45:00.675Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-03-20T14:45:00.675Z DEBUG [conditions] conditions/conditions.go:98 New condition regexp: map[]
2020-03-20T14:45:00.675Z DEBUG [conditions] conditions/conditions.go:98 New condition !regexp: map[]
2020-03-20T14:45:00.675Z DEBUG [processors] processors/processor.go:101 Generated new processors: add_locale=[format=offset], condition=!regexp: map[]
2020-03-20T14:45:00.675Z DEBUG [input] log/config.go:204 recursive glob enabled
2020-03-20T14:45:00.676Z DEBUG [input] log/input.go:164 exclude_files: [(?-s:.)gz(?-m:$)]. Number of stats: 0
2020-03-20T14:45:00.676Z DEBUG [input] log/input.go:185 input with previous states loaded: 0
2020-03-20T14:45:00.676Z INFO log/input.go:152 Configured paths: [/var/log/logstash/logstash-slowlog-plain*.log]
2020-03-20T14:45:00.676Z INFO input/input.go:114 Starting input of type: log; ID: 16452081253528920147
2020-03-20T14:45:00.676Z DEBUG [cfgfile] cfgfile/reload.go:133 Checking module configs from: /usr/share/filebeat/modules.d/*.yml
2020-03-20T14:45:00.676Z DEBUG [input] log/input.go:191 Start next scan
2020-03-20T14:45:00.676Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-03-20T14:45:00.676Z DEBUG [cfgfile] cfgfile/reload.go:147 Number of module configs found: 0
2020-03-20T14:45:00.676Z INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 2
2020-03-20T14:45:00.676Z INFO cfgfile/reload.go:171 Config reloader started
2020-03-20T14:45:00.677Z DEBUG [cfgfile] cfgfile/reload.go:197 Scan for new config files
2020-03-20T14:45:00.677Z DEBUG [cfgfile] cfgfile/reload.go:216 Number of module configs found: 0
2020-03-20T14:45:00.677Z DEBUG [reload] cfgfile/list.go:62 Starting reload procedure, current runners: 0
2020-03-20T14:45:00.677Z DEBUG [reload] cfgfile/list.go:80 Start list: 0, Stop list: 0
2020-03-20T14:45:00.677Z INFO cfgfile/reload.go:226 Loading of config files completed.
2020-03-20T14:45:10.679Z DEBUG [input] input/input.go:152 Run input
2020-03-20T14:45:10.679Z DEBUG [input] log/input.go:191 Start next scan
2020-03-20T14:45:10.679Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-03-20T14:45:10.679Z DEBUG [input] input/input.go:152 Run input
2020-03-20T14:45:10.679Z DEBUG [input] log/input.go:191 Start next scan
2020-03-20T14:45:10.679Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-03-20T14:45:20.645Z DEBUG [input] input/input.go:152 Run input
2020-03-20T14:45:20.645Z DEBUG [input] log/input.go:191 Start next scan
2020-03-20T14:45:20.645Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-03-20T14:45:20.645Z DEBUG [input] input/input.go:152 Run input
2020-03-20T14:45:20.646Z DEBUG [input] log/input.go:191 Start next scan
2020-03-20T14:45:20.646Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-03-20T14:45:30.639Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":{"ms":48}},"total":{"ticks":100,"time":{"ms":108},"value":100},"user":{"ticks":60,"time":{"ms":60}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":6},"info":{"ephemeral_id":"1c95e057-60a4-4d75-b15e-4d4b184c9b54","uptime":{"ms":30046}},"memstats":{"gc_next":8162320,"memory_alloc":5191688,"memory_total":13367280,"rss":48828416},"runtime":{"goroutines":28}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"type":"logstash"},"pipeline":{"clients":2,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":6},"load":{"1":0.04,"15":0,"5":0.04,"norm":{"1":0.0067,"15":0,"5":0.0067}}}}}}