Sending logs from OpenShift using Filebeat to external Elastic Stack

vivek14 · May 30, 2019, 4:37am

Hi,

I want to install Beats (Filebeat for time being) on OpenShift in order to ship data to Logstash. This Logstash instance is part of external Elastic Stack environment outside of OpenShift.

Is this currently possible?

Any pointers/ help is greatly appreciated.

The reasons for using external Elastic Stack are

I would like to have full control over ELK components. e.g. custom log filtering in Logstash
Ability to install version of ELK instead of old versions that come with OpenShift.

Thanks in advance.

faec · May 31, 2019, 7:27pm

Yes, filebeat should be usable standalone as long as it can still communicate with the Logstash / elastic endpoint you're using.

vivek14 · June 3, 2019, 3:46am

Thanks for responding.

I installed Filebeat using the template (filebeat-kubernetes_original.yaml) mentioned in documentation. Changed few things (version to 6.5.4, host path to /var/lib/filebeat-data/registry/filebeat, etc). I also deployed a sample php app to verify if Filebeat is harvesting logs from this sample app or not. It seems the filebeat pod is not harvesting logs from this app.

Here's log from filebeat pod

2019-06-03T03:41:27.099Z	INFO	instance/beat.go:592	Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2019-06-03T03:41:27.100Z	INFO	instance/beat.go:599	Beat UUID: 083d14d5-55e0-43d6-93a3-fa933e383140
2019-06-03T03:41:27.100Z	INFO	[seccomp]	seccomp/seccomp.go:116	Syscall filter successfully installed
2019-06-03T03:41:27.100Z	INFO	[beat]	instance/beat.go:825	Beat info	{"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "083d14d5-55e0-43d6-93a3-fa933e383140"}}}
2019-06-03T03:41:27.100Z	INFO	[beat]	instance/beat.go:834	Build info	{"system_info": {"build": {"commit": "bd8922f1c7e93d12b07e0b3f7d349e17107f7826", "libbeat": "6.5.4", "time": "2018-12-17T20:22:29.000Z", "version": "6.5.4"}}}
2019-06-03T03:41:27.100Z	INFO	[beat]	instance/beat.go:837	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.10.6"}}}
2019-06-03T03:41:27.104Z	INFO	[beat]	instance/beat.go:841	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-06-03T01:02:01Z","containerized":true,"name":"filebeat-h9dnr","ip":["127.0.0.1/8","::1/128","172.17.0.8/16","fe80::42:acff:fe11:8/64"],"kernel_version":"3.10.0-957.5.1.el7.x86_64","mac":["02:42:ac:11:00:08"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":6,"patch":1810,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0}}}
2019-06-03T03:41:27.105Z	INFO	[beat]	instance/beat.go:870	Process info	{"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 1, "ppid": 0, "seccomp": {"mode":"filter"}, "start_time": "2019-06-03T03:41:26.350Z"}}}
2019-06-03T03:41:27.105Z	INFO	instance/beat.go:278	Setup Beat: filebeat; Version: 6.5.4
2019-06-03T03:41:30.106Z	INFO	add_cloud_metadata/add_cloud_metadata.go:319	add_cloud_metadata: hosting provider type not detected.
2019-06-03T03:41:30.107Z	INFO	elasticsearch/client.go:163	Elasticsearch url: http://192.168.0.38:9200 
2019-06-03T03:41:30.108Z	INFO	[publisher]	pipeline/module.go:110	Beat name: filebeat-h9dnr
2019-06-03T03:41:30.109Z	INFO	instance/beat.go:400	filebeat start running.
2019-06-03T03:41:30.110Z	INFO	registrar/registrar.go:134	Loading registrar data from /usr/share/filebeat/data/registry
2019-06-03T03:41:30.110Z	INFO	registrar/registrar.go:141	States Loaded from registrar: 0
2019-06-03T03:41:30.110Z	INFO	crawler/crawler.go:72	Loading Inputs: 0
2019-06-03T03:41:30.110Z	INFO	crawler/crawler.go:106	Loading and starting Inputs completed. Enabled inputs: 0
2019-06-03T03:41:30.110Z	WARN	[cfgwarn]	kubernetes/kubernetes.go:51	BETA: The kubernetes autodiscover is beta
2019-06-03T03:41:30.111Z	INFO	kubernetes/util.go:86	kubernetes: Using pod name filebeat-h9dnr and namespace elastic to discover kubernetes node
2019-06-03T03:41:30.111Z	INFO	cfgfile/reload.go:150	Config reloader started
2019-06-03T03:41:30.112Z	INFO	cfgfile/reload.go:205	Loading of config files completed.
2019-06-03T03:41:30.150Z	INFO	kubernetes/util.go:93	kubernetes: Using node localhost discovered by in cluster pod node query
2019-06-03T03:41:30.150Z	WARN	[cfgwarn]	hints/logs.go:56	BETA: The hints builder is beta
2019-06-03T03:41:30.151Z	INFO	autodiscover/autodiscover.go:103	Starting autodiscover manager
2019-06-03T03:41:30.151Z	INFO	kubernetes/watcher.go:180	kubernetes: Performing a resource sync for *v1.PodList
2019-06-03T03:41:30.170Z	INFO	kubernetes/watcher.go:194	kubernetes: Resource sync done
2019-06-03T03:41:30.171Z	INFO	kubernetes/watcher.go:238	kubernetes: Watching API for resource events
2019-06-03T03:41:30.173Z	INFO	log/input.go:138	Configured paths: [/var/lib/docker/containers/50fd2f66271cb639004b4045e57fcf4c9e744fda25873c4bd9f8a8e674f03c04/*.log]
2019-06-03T03:41:30.173Z	INFO	input/input.go:114	Starting input of type: docker; ID: 10845280166695259504 
2019-06-03T03:41:30.174Z	INFO	log/input.go:138	Configured paths: [/var/lib/docker/containers/e96be9f281f4682a176702c7bec44942cb8c319bcbfce9672ec16e85c2e8dd66/*.log]
2019-06-03T03:41:30.174Z	INFO	input/input.go:114	Starting input of type: docker; ID: 5371052876786438217

Did I miss anything? Could you help?

Thanks

system · July 1, 2019, 3:46am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.