Filebeat to collect the logs from kubernetes cluster running on AWS

Elastic Stack Beats
1

Hi there,

My elasticsearch and kibana instance are running on on-prem and the k8s cluster where i would like to collect the logs from is running on AWS.

I cannot have filebeat running inside the kubernetes cluster as pushing data from AWS to elasticsearch (running on on-prem is not possible ). but it would be great if i can have filebeat running on any of the on-prem server and pointing it to the pull logs from remote k8s cluster. but i am not sure if filebeat is having capability to pull the logs from remote k8scluster?

Please suggest the best way to pull the logs from k8s cluster running on AWS to elasticsearch running on-prem ?

Thanks

2

It can not collect logs remotely. You should deploy it as a DaemonSet as outlined in this blog post and have it ship logs to the remote cluster.

3

Thanks for the response. I have tried this but the only issue is elasticsearch cluster is running on on-prem , so i cannot push the logs from file beat running on AWS to on-prem cluster because of the network settings.

are we planning to have the support for filebeat to pull the logs from remote server in the future releases? If yes , i can simply install filebeat on on-prem server and configure that to pull the logs from AWS k8s (as reaching out to public network from on-prem server is possible)

and kindly suggest if there is any better way to overcome this issue?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.