# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=190 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 191 --- ## [\[Filebeat\] Define custom Ingest Node for Kafka Output](https://discuss.elastic.co/t/filebeat-define-custom-ingest-node-for-kafka-output/259032)
**Author:** [@dacamposol](https://discuss.elastic.co/u/dacamposol)\ **Replies:** 2\ **Last updated:** [December 18, 2020, 9:48am UTC](https://discuss.elastic.co/t/filebeat-define-custom-ingest-node-for-kafka-output/259032 "2020-12-18T09:48:43Z")
Good afternoon guys! I have a question regarding the configuration of the pipelines and Filebeat, when we're using a Kafka as output: When we have ElasticSearch as output, we can just define the pipeline we want to use… --- ## [Should i configured xpack on filebeat?](https://discuss.elastic.co/t/should-i-configured-xpack-on-filebeat/259059)
**Author:** [@Taufik\_Maulana](https://discuss.elastic.co/u/Taufik_Maulana)\ **Replies:** 1\ **Last updated:** [December 18, 2020, 9:03am UTC](https://discuss.elastic.co/t/should-i-configured-xpack-on-filebeat/259059 "2020-12-18T09:03:13Z")
Hi ! I have a question, should i configured xpack on filebeat ? or just in Elasticsearch ? because according to this link it says Verify that the xpack.security.enabled setting is true on each node in your cluster. .… --- ## [Filebeat S3 input support for AWS WAF logs (support application/octet-stream)](https://discuss.elastic.co/t/filebeat-s3-input-support-for-aws-waf-logs-support-application-octet-stream/258981)
**Author:** [@rubal033](https://discuss.elastic.co/u/rubal033)\ **Replies:** 1\ **Last updated:** [December 18, 2020, 1:54am UTC](https://discuss.elastic.co/t/filebeat-s3-input-support-for-aws-waf-logs-support-application-octet-stream/258981 "2020-12-18T01:54:03Z")
As of now, there is Filebeat s3 input that doesn't support AWS WAF logs. AWS WAF logs use Kinesis Firehose to get to S3 and the "Content-type" is set to "application/octet-stream". Due to this, the logs didn't get expand… --- ## [Squid Module: Splitting multiple logs which are combined as a single message in Kibana](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609)
**Author:** [@ron\_g](https://discuss.elastic.co/u/ron_g)\ **Replies:** 4\ **Last updated:** [December 17, 2020, 3:24pm UTC](https://discuss.elastic.co/t/squid-module-splitting-multiple-logs-which-are-combined-as-a-single-message-in-kibana/258609 "2020-12-17T15:24:09Z")
We have the problem that all logs which are send from our Squid server over Filebeat with the Squid module are combined as single messages in Kibana. The message contains 5 up to 15 entrys which are in fact single log l… --- ## [Filebeat Cisco Umbrella](https://discuss.elastic.co/t/filebeat-cisco-umbrella/258779)
**Author:** [@nathanachey](https://discuss.elastic.co/u/nathanachey)\ **Replies:** 3\ **Last updated:** [December 17, 2020, 2:01pm UTC](https://discuss.elastic.co/t/filebeat-cisco-umbrella/258779 "2020-12-17T14:01:26Z")
I'm trying to setup Filebeat to use the Cisco Umbrella module. We have the Cisco maintained bucket. The URL Cisco provides is s3://cisco-managed-us-west-1/2613934\_1b0f4f029c8f0b75a2f9a6d4e06a79d6cbbc41bb The error I'm… --- ## [Error elasticsearch/client.go:407](https://discuss.elastic.co/t/error-elasticsearch-client-go-407/258825)
**Author:** [@nuut](https://discuss.elastic.co/u/nuut)\ **Replies:** 1\ **Last updated:** [December 17, 2020, 11:08am UTC](https://discuss.elastic.co/t/error-elasticsearch-client-go-407/258825 "2020-12-17T11:08:48Z")
hi team i ge error like this Dec 16 14:25:16 localhost filebeat\[33713\]: 2020-12-16T14:25:15.598+0700 WARN \[elasticsearch\] elasticsearch/client.go:407 Cannot index event publisher.Event can y… --- ## [Supported output types for elastic agent?](https://discuss.elastic.co/t/supported-output-types-for-elastic-agent/258901)
**Author:** [@slmingol](https://discuss.elastic.co/u/slmingol)\ **Replies:** 1\ **Last updated:** [December 17, 2020, 8:56am UTC](https://discuss.elastic.co/t/supported-output-types-for-elastic-agent/258901 "2020-12-17T08:56:29Z")
Is it safe to assume that given Elastic Agent is built on top of \*beats technology that it'll support the use of different output types such as Kafka? https://www.elastic.co/guide/en/beats/filebeat/master/kafka-output.… --- ## [Filebeats with metricbeats data in standalone cluster](https://discuss.elastic.co/t/filebeats-with-metricbeats-data-in-standalone-cluster/258841)
**Author:** [@mutt13y](https://discuss.elastic.co/u/mutt13y)\ **Replies:** 2\ **Last updated:** [December 17, 2020, 1:40am UTC](https://discuss.elastic.co/t/filebeats-with-metricbeats-data-in-standalone-cluster/258841 "2020-12-17T01:40:55Z")
Hello, I am trying to move over to metricbeat monitoring running 7.10 eveywhere I have Filebeat -\> logstash -\> elastic main cluster filebeat node metricbeat -\> monitoring cluster logstash node metricbeat -\> monitorin… --- ## [Docker Metricbeat 7.6.0 doesn't send Host processes](https://discuss.elastic.co/t/docker-metricbeat-7-6-0-doesnt-send-host-processes/257684)
**Author:** [@gregbk](https://discuss.elastic.co/u/gregbk)\ **Replies:** 2\ **Last updated:** [December 17, 2020, 1:38am UTC](https://discuss.elastic.co/t/docker-metricbeat-7-6-0-doesnt-send-host-processes/257684 "2020-12-17T01:38:26Z")
Hello ELK community! I tried to run a basic setup with ELK version 7.6.0 Metricbeat.yml metricbeat.config: modules: path: ${path.config}/modules.d/\*.yml # Reload module configs as they change: reload.ena… --- ## [Filebeat: Unexpected file opening error: file info is not identical with opened file. Aborting harvesting and retrying file later again](https://discuss.elastic.co/t/filebeat-unexpected-file-opening-error-file-info-is-not-identical-with-opened-file-aborting-harvesting-and-retrying-file-later-again/258315)
**Author:** [@elk\_follower](https://discuss.elastic.co/u/elk_follower)\ **Replies:** 8\ **Last updated:** [December 16, 2020, 8:43pm UTC](https://discuss.elastic.co/t/filebeat-unexpected-file-opening-error-file-info-is-not-identical-with-opened-file-aborting-harvesting-and-retrying-file-later-again/258315 "2020-12-16T20:43:51Z")
Hello, We have ELK 7.8.1 stack running in K8 cluster. Filebeat on one of the node is not able to keep up with the logs generated by one MS. This was working fine until couple of days. Missing logs are from a container … --- ## [Metricbeat Process information](https://discuss.elastic.co/t/metricbeat-process-information/258778)
**Author:** [@jchaves506](https://discuss.elastic.co/u/jchaves506)\ **Replies:** 5\ **Last updated:** [December 16, 2020, 5:52pm UTC](https://discuss.elastic.co/t/metricbeat-process-information/258778 "2020-12-16T17:52:58Z")
is there a way to get the version of the processes that metricbeat gathers with the system's module? or do I need to use a different module?, I'm trying to get all the information about the executable file running at an … --- ## [Filebeat not sending logs anymore](https://discuss.elastic.co/t/filebeat-not-sending-logs-anymore/257246)
**Author:** [@toms130](https://discuss.elastic.co/u/toms130)\ **Replies:** 1\ **Last updated:** [December 16, 2020, 10:56am UTC](https://discuss.elastic.co/t/filebeat-not-sending-logs-anymore/257246 "2020-12-16T10:56:55Z")
Hi there, I've gor problem with a filebeat instance which ends to stop sending logs after smome time. Here is my setup : type: log enabled: true paths: - /opt/TalendRemoteEngine/TalendJobServersFiles/jobexecu… --- ## [Metricbeat AWS Module estimated costs](https://discuss.elastic.co/t/metricbeat-aws-module-estimated-costs/257863)
**Author:** [@Jurilz](https://discuss.elastic.co/u/Jurilz)\ **Replies:** 2\ **Last updated:** [December 16, 2020, 10:43am UTC](https://discuss.elastic.co/t/metricbeat-aws-module-estimated-costs/257863 "2020-12-16T10:43:53Z")
Good day, I'm thinking about the using metricbeats AWS module to collect and display metrics of one or several EC2 instances. And therefore I have to estimate the costs of AWS charges on CloudWatch API requests. Let's … --- ## [Winlogbeat service failing to start](https://discuss.elastic.co/t/winlogbeat-service-failing-to-start/258414)
**Author:** [@Craig2188](https://discuss.elastic.co/u/Craig2188)\ **Replies:** 2\ **Last updated:** [December 16, 2020, 10:13am UTC](https://discuss.elastic.co/t/winlogbeat-service-failing-to-start/258414 "2020-12-16T10:13:26Z")
Hi, I have installed Winlogbeat 7.9.2 onto a 2008 32bit server via the ps1 file. I go to start the service and get the following error: The winlogbeat service failed to start due to the following error: winlogbeat is … --- ## [Filebeat not able to send all logs](https://discuss.elastic.co/t/filebeat-not-able-to-send-all-logs/257930)
**Author:** [@user2416](https://discuss.elastic.co/u/user2416)\ **Replies:** 11\ **Last updated:** [December 16, 2020, 9:47am UTC](https://discuss.elastic.co/t/filebeat-not-able-to-send-all-logs/257930 "2020-12-16T09:47:12Z")
Hi, We are using filebeat as a deamonset on kubernetes nodes to collect all application logs and sending them to logstash and then to elasticsearch. In our Prod environment, we have some pods writing huge volume of eve… --- ## [Filebeats processor active not right under specific input](https://discuss.elastic.co/t/filebeats-processor-active-not-right-under-specific-input/258816)
**Author:** [@dukeyang](https://discuss.elastic.co/u/dukeyang)\ **Replies:** 0\ **Last updated:** [December 16, 2020, 8:16am UTC](https://discuss.elastic.co/t/filebeats-processor-active-not-right-under-specific-input/258816 "2020-12-16T08:16:07Z")
In the official doc, it says: Processors are valid: At the top-level in the configuration. The processor is applied to all data collected by Filebeat. Under a specific input. The processor is applied to the data colle… --- ## [Separate Filebeat container vs Filebeat on every container architectural decision](https://discuss.elastic.co/t/separate-filebeat-container-vs-filebeat-on-every-container-architectural-decision/258557)
**Author:** [@thanastasiadis](https://discuss.elastic.co/u/thanastasiadis)\ **Replies:** 1\ **Last updated:** [December 16, 2020, 9:10am UTC](https://discuss.elastic.co/t/separate-filebeat-container-vs-filebeat-on-every-container-architectural-decision/258557 "2020-12-16T09:10:38Z")
In a micro-service architecture, I am trying to decide between the two architectures: Installing file beat on every container Run a separate file beat container The official documentation suggests to install filebeat … --- ## [Why kubernetes' statefulset didn't run evenly for 3 pods?](https://discuss.elastic.co/t/why-kubernetes-statefulset-didnt-run-evenly-for-3-pods/258797)
**Author:** [@iooi](https://discuss.elastic.co/u/iooi)\ **Replies:** 2\ **Last updated:** [December 16, 2020, 8:43am UTC](https://discuss.elastic.co/t/why-kubernetes-statefulset-didnt-run-evenly-for-3-pods/258797 "2020-12-16T08:43:56Z")
I deployed a logstash by statefulset kind with 3 replicas in k8s. Using filebeat to send data to it. --- apiVersion: apps/v1 kind: StatefulSet metadata: name: logstash-nginx spec: serviceName: "logstash" selector:… --- ## [How to use json logs and also multiline together](https://discuss.elastic.co/t/how-to-use-json-logs-and-also-multiline-together/258347)
**Author:** [@user2416](https://discuss.elastic.co/u/user2416)\ **Replies:** 3\ **Last updated:** [December 16, 2020, 8:33am UTC](https://discuss.elastic.co/t/how-to-use-json-logs-and-also-multiline-together/258347 "2020-12-16T08:33:03Z")
Hi, We are using multiline pattern in filebeat but we also have Json logs which doesnt match that multiline pattern. We are seeing both multiline logs and json logs but some json logs are missing and also json logs are … --- ## [Why does functionbeat for AWS need to be able to make Cloudformation actions?](https://discuss.elastic.co/t/why-does-functionbeat-for-aws-need-to-be-able-to-make-cloudformation-actions/258647)
**Author:** [@Jonathan\_Detert](https://discuss.elastic.co/u/Jonathan_Detert)\ **Replies:** 1\ **Last updated:** [December 16, 2020, 1:52am UTC](https://discuss.elastic.co/t/why-does-functionbeat-for-aws-need-to-be-able-to-make-cloudformation-actions/258647 "2020-12-16T01:52:51Z")
The IAM permissions doc for functionbeat lists a bunch of Cloudformation actions to be granted. What are they needed for? For the deployment process only, or something else? Thanks --- ## [Metricbeat AWS RDS Module not retrieving db\_instance.identifiers](https://discuss.elastic.co/t/metricbeat-aws-rds-module-not-retrieving-db-instance-identifiers/258174)
**Author:** [@achalamalasetti](https://discuss.elastic.co/u/achalamalasetti)\ **Replies:** 2\ **Last updated:** [December 16, 2020, 1:51am UTC](https://discuss.elastic.co/t/metricbeat-aws-rds-module-not-retrieving-db-instance-identifiers/258174 "2020-12-16T01:51:12Z")
we are using metricbeat (7.10.0) -\> AWS module, RDS metrics. The metric data that is pulled looks incomplete. it doesn't retrieve aws.rds.db\_instance.engine\_name for some and for some it doesn't retrieve aws.rds.db\_inst… --- ## [Error getting group status](https://discuss.elastic.co/t/error-getting-group-status/258725)
**Author:** [@SVictor](https://discuss.elastic.co/u/SVictor)\ **Replies:** 1\ **Last updated:** [December 16, 2020, 12:13am UTC](https://discuss.elastic.co/t/error-getting-group-status/258725 "2020-12-16T00:13:52Z")
Hello, can anyone help me with metricbeats. I have a kubernetes cluster 1.16 and I installed metricbeat, it work but i got 1 error: instance/metrics.go:285 error getting group status: open /proc/5806/cgroup: no such fi… --- ## [Enable more Metrics](https://discuss.elastic.co/t/enable-more-metrics/258764)
**Author:** [@jsahiwal](https://discuss.elastic.co/u/jsahiwal)\ **Replies:** 3\ **Last updated:** [December 15, 2020, 11:32pm UTC](https://discuss.elastic.co/t/enable-more-metrics/258764 "2020-12-15T23:32:59Z")
Metrics Module What do we need to enable to see the following metrics System Paging FileSystem Paging Process Content Switches Forks Filesystems Inodes ( Total, Used, Free) --- ## ["Ignore\_older" the other way around](https://discuss.elastic.co/t/ignore-older-the-other-way-around/258653)
**Author:** [@jchaves506](https://discuss.elastic.co/u/jchaves506)\ **Replies:** 2\ **Last updated:** [December 15, 2020, 8:21pm UTC](https://discuss.elastic.co/t/ignore-older-the-other-way-around/258653 "2020-12-15T20:21:14Z")
Hello does anyone knows how can I only read files older\_than = XX, something like an include\_older. Right now I'm reading files with name format log\_date.txt, the problem is that I don't want to be reading files that a… --- ## [Suricata module - no parsing for XFF field (x forward ip)](https://discuss.elastic.co/t/suricata-module-no-parsing-for-xff-field-x-forward-ip/258641)
**Author:** [@sportelh](https://discuss.elastic.co/u/sportelh)\ **Replies:** 4\ **Last updated:** [December 15, 2020, 6:02pm UTC](https://discuss.elastic.co/t/suricata-module-no-parsing-for-xff-field-x-forward-ip/258641 "2020-12-15T18:02:07Z")
Hi Guys, I am playing arround with ELK for suricata logs, to see if it is suitable for my employer I have created some nice dashboards (kibana) so far so good. But to my surprise there is no suricata.eve field for th… --- ## [Not getting kubernetes metadata in my logs](https://discuss.elastic.co/t/not-getting-kubernetes-metadata-in-my-logs/258621)
**Author:** [@jknott](https://discuss.elastic.co/u/jknott)\ **Replies:** 2\ **Last updated:** [December 15, 2020, 3:52pm UTC](https://discuss.elastic.co/t/not-getting-kubernetes-metadata-in-my-logs/258621 "2020-12-15T15:52:34Z")
Here is how I have filebeat helm chart configured. What am I missing? processors: - add\_cloud\_metadata: - add\_kubernetes\_metadata: host: ${NODE\_NAME} matchers: - l… --- ## [Error with filebeat on kubernetes 1.20 using add\_kubernetes\_metadata](https://discuss.elastic.co/t/error-with-filebeat-on-kubernetes-1-20-using-add-kubernetes-metadata/258722)
**Author:** [@yoannma](https://discuss.elastic.co/u/yoannma)\ **Replies:** 0\ **Last updated:** [December 15, 2020, 2:20pm UTC](https://discuss.elastic.co/t/error-with-filebeat-on-kubernetes-1-20-using-add-kubernetes-metadata/258722 "2020-12-15T14:20:39Z")
Hello, I installed filebeat 7.10.1 on a kubernetes cluster 1.20 using the DaemonSet from https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat/filebeat-configmap.yaml Using the hints based autodiscov… --- ## [Filebeat Pod restarting continuously due to OOM](https://discuss.elastic.co/t/filebeat-pod-restarting-continuously-due-to-oom/258704)
**Author:** [@pavank](https://discuss.elastic.co/u/pavank)\ **Replies:** 1\ **Last updated:** [December 15, 2020, 12:44pm UTC](https://discuss.elastic.co/t/filebeat-pod-restarting-continuously-due-to-oom/258704 "2020-12-15T12:44:45Z")
Hi All, We have deployed Filebeat as a pod in our environment. We have set memory limit of 2GB in the deployment resources. We still see the pods getting restarted almost every day. Attaching the configmap and deploym… --- ## [Integrate cisco devices](https://discuss.elastic.co/t/integrate-cisco-devices/258707)
**Author:** [@sbathla](https://discuss.elastic.co/u/sbathla)\ **Replies:** 1\ **Last updated:** [December 15, 2020, 11:41am UTC](https://discuss.elastic.co/t/integrate-cisco-devices/258707 "2020-12-15T11:41:18Z")
Hi All, I am new to elasticstack. I want to integrate Cisco devices with elasticsearch and kibana for which cisco module under filebeat is available for integration. But filebeat is installed on the host which has to be… --- ## [Metricbeat only run system module instead of running both system and apache module](https://discuss.elastic.co/t/metricbeat-only-run-system-module-instead-of-running-both-system-and-apache-module/258098)
**Author:** [@Minh\_Ti\_n\_Tr\_n](https://discuss.elastic.co/u/Minh_Ti_n_Tr_n)\ **Replies:** 16\ **Last updated:** [December 15, 2020, 9:05am UTC](https://discuss.elastic.co/t/metricbeat-only-run-system-module-instead-of-running-both-system-and-apache-module/258098 "2020-12-15T09:05:20Z")
Hi all, I configured metricbeat on my server for collecting both system & apache data with default module but only system data from host showed on kibana. The apache dashboard is empty data and don't know the reason. h… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=189) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=191)