# How metricbeat to monitor Filebeat running in Daemonset on K8s **URL:** \ **Category:** Beats\ **Tags:** docker, filebeat, metricbeat\ **Created:** [March 11, 2022, 5:01am UTC](https://discuss.elastic.co/t/how-metricbeat-to-monitor-filebeat-running-in-daemonset-on-k8s/299397 "2022-03-11T05:01:13Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![cmy214](https://avatars.discourse-cdn.com/v4/letter/c/a8b319/32.png) [@cmy214](https://discuss.elastic.co/u/cmy214)\ **Post date:** [March 11, 2022, 5:01am UTC](https://discuss.elastic.co/t/how-metricbeat-to-monitor-filebeat-running-in-daemonset-on-k8s/299397/1 "2022-03-11T05:01:13Z")
Hi all, My **filebeat** is in **Daemonset** on K8s cluster. In 7.17 internal monitoring of beats is removed and requires a separate metricbeat for achieving monitoring. So I'm setting up **metricbeat as Daemonset to monitor my filebeat**. Problem: 1. My metricbeat does not know my filebeat's endpoint. **Creating a K8s Service** for my filebeat daemonset **only reach a random filebeat pod** (no way to reach specific node). But I need to monitor all filebeat running on every node. 2. Correct me if I am wrong. **Autodiscover requires privileged mode of SCC** (i.e. higher permission) which is not allowed to grant in my organization. 3. I explored creating a headless service for my filebeat and returns a list of IPs. However I checked some articles seems that **metricbeat does not monitor multiple IPs returned** from an DNS. The same, I want to use metricbeat to monitor my logstash instances running as Deployment on K8s as well. Thanks. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [March 11, 2022, 5:15am UTC](https://discuss.elastic.co/t/how-metricbeat-to-monitor-filebeat-running-in-daemonset-on-k8s/299397/2 "2022-03-11T05:15:33Z")
Hi @cmy214 Welcome to the community! I think you're perhaps a bit confused. You don't need metricbeat To monitor filebeat. What was removed was an old legacy collection method. There is still internal collection which we use all the time to ship filebeat metrics to Elasticsearch > **[Use internal collection to send monitoring data | Filebeat Reference \[7.17\] |...](https://www.elastic.co/guide/en/beats/filebeat/7.17/monitoring-internal-collection.html#monitoring-internal-collection)** Just point the internal collection to to the cluster where you want the metrics to show up. In fact, if you just add these two lines, it will send the metrics to the same cluster that your Elasticsearch output is set to. ```auto monitoring: enabled: true ``` Perhaps I'm confused but we use this today and other kubernetes environment. ---
**Author:** ![cmy214](https://avatars.discourse-cdn.com/v4/letter/c/a8b319/32.png) [@cmy214](https://discuss.elastic.co/u/cmy214)\ **Post date:** [March 11, 2022, 7:38am UTC](https://discuss.elastic.co/t/how-metricbeat-to-monitor-filebeat-running-in-daemonset-on-k8s/299397/3 "2022-03-11T07:38:22Z")
hi @stephenb, thanks for your information, I understand now. However, metricbeat is still a better option because 1. More comprehensive data can be collected by metricbeat 2. I understand that the roadmap of internal monitoring will shift to metricbeat [[Stack Monitoring] Remove internal collectors · Issue #11169 · elastic/logstash · GitHub](https://github.com/elastic/logstash/issues/11169) Could you / anyone assist me, please? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [March 11, 2022, 2:37pm UTC](https://discuss.elastic.co/t/how-metricbeat-to-monitor-filebeat-running-in-daemonset-on-k8s/299397/4 "2022-03-11T14:37:02Z")
The GitHub issue you reference is not related to filebeat and metrics collection. It's an old logstash issue. Can you show me where it says more comprehensive metricset for filebeat? As far as I know, it's the exact same same metricset whether you internal collection or metricbeat. Yes if you want to collect container metrics as well as filebeat metrics absolutely you should use metricbeat. If you want to deploy metric beat in kubernetes, I would start with this. > **[Run Metricbeat on Kubernetes | Metricbeat Reference \[8.1\] | Elastic](https://www.elastic.co/guide/en/beats/metricbeat/current/running-on-kubernetes.html)** I would get metric beat all running for the normal system and container metrics. Then you'll need to do all the steps under the metricbest collection. Again, I have a number of customers running large kubernetes clusters with filebeat Internal collection and it works great. What most people do is deploy metricbeat to collect all the system and kubernetes and container metrics etc. Then for all the beats including even metricbeat, they use the internal collection to understand the health of their beats. That's the normal pattern that I see deployed in production. ---
**Author:** ![cmy214](https://avatars.discourse-cdn.com/v4/letter/c/a8b319/32.png) [@cmy214](https://discuss.elastic.co/u/cmy214)\ **Post date:** [March 17, 2022, 3:53am UTC](https://discuss.elastic.co/t/how-metricbeat-to-monitor-filebeat-running-in-daemonset-on-k8s/299397/5 "2022-03-17T03:53:13Z")
Hi stephenb, Yes I would like to collect container metrics as well. So I want to use metricbeat. Following your doc, it requires higher privilege below in OpenShift which is the exact same issue that I am encountering (my problem pt. #2). My organzation does not allow. securityContext: runAsUser: 0 privileged: true Regards, Tom ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [March 17, 2022, 5:23am UTC](https://discuss.elastic.co/t/how-metricbeat-to-monitor-filebeat-running-in-daemonset-on-k8s/299397/6 "2022-03-17T05:23:42Z")
Hi @cmy214 I am not an openshift expert but I know there has been some discussion about it running metricbeat in restricted environments but I do not know much about it. Here is the issue I believe. > > > In general, monitoring systems with Beats requires certain privileges, but they …can work with limited functionality without them. > > This issue keeps track of known issues running Beats on Kubernetes restricted environments, to explore alternatives that would allow to use more functionality with less privileges. Some of the issues will probably apply to other environments too. > > Restricted environments have some limitations, for example the \[restricted Security Context Constraints in Openshift\](https://docs.openshift.com/container-platform/3.6/admin\_guide/manage\_scc.html#examining-a-security-context-constraints-object) include: > \* Privileged mode not allowed. > \* Some capabilities are forbidden. > \* Limited set of volumes allowed (no \`hostPath\`). > \* Host namespaces not allowed. > \* User ID must be in a range of ids defined in the project (so the uid will be unknown beforehand, and \`runAsUser: 0\` is not allowed). > > Known issues > === > > \*\*Data persistence\*\* > > Data persistence is required in beats in some features: > \* \`meta.json\` (with stored UUID, used in stack monitoring?) > \* Filebeat registry > \* Auditbeat datastores (not always required but recommended) > \* \[\[TBD\](https://github.com/elastic/beats/issues/19511)\] Metadata on-disk caches > > Beats reference manifests currently rely on being able to create a directory in the host with \`hostPath\`. This is not possible in restricted environments because \`hostPath\` is not allowed, and because directories created by \`hostPath\` can only be written by root (uid 0 on host). > > Possible improvements/alternatives: > \* Document alternatives with persistent volume claims (will depend on available volume providers) > \* Explore deployment as \`StatefulSet\` on cases where \`DaemonSet\` is not needed. > \* Depend on the \[local storage operator\](https://docs.openshift.com/container-platform/4.4/storage/persistent\_storage/persistent-storage-local.html). > \* Use an external database to store data instead of files. > > \*\*Auditing\*\* > > Auditing with auditbeat requires to be run with audit capabilities, on host pid namespace and with uid 0. > > There is probably no possible alternative, but we could do some improvements: > \* Explicitly document these requirements somewhere. > \* Offer alternative configurations for Auditbeat to use other features that don't require so much privileges. > > \*\*Host metadata\*\* > > Retrieving host metadata requires to run beats on host namespace, this is not allowed on restricted environments. > > Possible alternatives/improvements: > \* Document this limitation in the manifests, so affected parts can be commented out, or they are commented out by default. > \* Use information obtained from the downward API (\`spec.nodeName\` as the host name, though not always the same, \`status.hostIP\` as the host IP). > \* Look for alternative ways of collecting host information using existing APIs. > > \*\*Host network monitoring\*\* > > Monitoring the host network relies on running Beats on the host network namespace. > > Possible alternatives/improvements: > \* Add comments about this in the reference manifests/docs. > \* Comment out by default features that require running on host namespaces. > \* Try to collect more information from host filesystems like \`/proc\`. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 4, 2022, 8:34am UTC](https://discuss.elastic.co/t/how-metricbeat-to-monitor-filebeat-running-in-daemonset-on-k8s/299397/7 "2022-11-04T08:34:34Z")