We are running an ECK instance to gather logs from multiple K8s Clusters (not only the one it's running on), Winlogbeats agent on Windows VMs, and would like to send logs from network devices (syslog) to Elastic as well.
To achieve our current scenario Elastic is published with a LB service.
ECK is a way for us to host elastic service but we do not want to limit its usage to K8s workloads logs.
We need a syslog (UDP) endpoint and be able to parse this format.
We have tested different solutions but lack expertise to make an appropriate choice:
syslog-ng to ECK (run in K8s, with LB Service to expose UDP port)
We also managed to run Beats with the example of Palo Alto. The specification is simple but ECK lacks example because it focuses on deploying filebeats for K8s logs, and not inputs that would be external to the cluster.
# CRD to create beats with ECK (Pod(s))
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
# Unique name for a beat
name: panos-filebeat
# Same namespace as ECK
namespace: eck-elasticsearch
spec:
# type filebeat to use panos module https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-panw.html
type: filebeat
version: 7.12.0
# Name of ECK' s elastic cluster: it will configure every setting/credentials inside the Beats to connect to ECK
elasticsearchRef:
name: elasticsearch
# Same as above
kibanaRef:
name: kibana
# Actual configuration of the beat. Will generate into a secret that will be mounted inside the Pod
config:
filebeat.modules:
# Enable Palo Alto module that sets up appropriate log parsing from syslog udp input (default, udp/9001 on localhost)
- module: panw
panos:
enabled: true
# Make the UDP server listen on pod's IP
var.syslog_host: 0.0.0.0
output.elasticsearch:
# Automatic configuration (elasticsearchRef) will import default CA of ECK. However we have our own certificate deployed for external access to elastic. Only workaround is to disable ssl verification. https://github.com/elastic/beats/issues/8164#issuecomment-597845134
ssl.verification_mode: none
# Choice of Deployment or DaemonSet to deploy the beat. We choose deployment and according specs
deployment:
podTemplate:
spec:
# Configuration is mounted in /etc/beat.yaml, needs this to access the file
securityContext:
runAsUser: 0
# Firewall sends log in its configured TZ without putting the TZ information in the log. We have to make the pod work in the same TZ.
containers:
- name: filebeat
volumeMounts:
- name: tz-config
mountPath: /etc/localtime
volumes:
- name: tz-config
hostPath:
path: /usr/share/zoneinfo/Europe/Zurich
type: File
# LoadBalancer Service to expose PanOS Filebeat Syslog UDP input externally and not supported YET in ECK's beat api
# https://github.com/elastic/cloud-on-k8s/issues/3866 Beat add spec for external service
apiVersion: v1
kind: Service
metadata:
name: panos-beat-svc
spec:
ports:
- port: 9001
protocol: UDP
targetPort: 9001
selector:
beat.k8s.elastic.co/name: panos-filebeat
type: LoadBalancer
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.