Getting other pods nginx log to Beats in ECK

Elastic Stack Beats
1

I have followed the ECK documents to configure Elasticsearch, Kibana, logstash and filebeat on my kubernetes cluster (it's a k3s home cluster)
The Filebeat configuration I followed from here
then I create the following version:

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: jsc-filebeat
  namespace: jsc-ns
spec:
  type: filebeat
  version: 8.17.2
  config:
    filebeat.inputs:
      - type: filestream
        id: nginx-filestream-id
        enabled: true
        paths:
          - /var/log/nginx/access.log
        fields:
          nginx: true
    output.logstash:
      hosts: ["jsc-logstash-ls-beats:5044"]
  daemonSet:
    podTemplate:
      spec:
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        securityContext:
          runAsUser: 0
        containers:
          - name: filebeat
            volumeMounts:
              - name: varlognginx
                mountPath: /var/log/nginx
        volumes:
          - name: varlognginx
            hostPath:
              path: /var/log/nginx

If I create a /var/log/nginx/access.log file on a filebeat pod, I can see the index created and the data showing in Kibana. That confirms that the overall setup is ok.
What I am missing is how to get the nginx logs from a different pods (I have a small website on a pod in the same k8s cluster and namespace) to be seen by the daemonSet
Is it a permission that I am missing? It might be more of a k8s question but I would think more people have tried this too.

2

It looks like you've posted this question twice, I've replied to your other post here Sending nginx logs from other pods to an ECK elasticsearch via Beats - #2 by strawgate

3

Thank you. Apologies, it's my first time using this forum, my other post appeared as draft and when I edited it posted a new one. I'll mark this as closed.

1 Like