# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=94 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 95 --- ## [Referencing captured groups inside yaml file](https://discuss.elastic.co/t/referencing-captured-groups-inside-yaml-file/303337)
**Author:** [@Draken](https://discuss.elastic.co/u/Draken)\ **Replies:** 0\ **Last updated:** [April 27, 2022, 1:36am UTC](https://discuss.elastic.co/t/referencing-captured-groups-inside-yaml-file/303337 "2022-04-27T01:36:23Z")
Hello, I'm running the postgresql module in metricbeat with the statement metricset active, i want to be able to determine the type of the query (insert, update, select or delete), so i add an add\_fields processor with … --- ## [Docker logs includes unreadable in Kibana](https://discuss.elastic.co/t/docker-logs-includes-unreadable-in-kibana/303196)
**Author:** [@dev9](https://discuss.elastic.co/u/dev9)\ **Replies:** 4\ **Last updated:** [April 27, 2022, 1:56am UTC](https://discuss.elastic.co/t/docker-logs-includes-unreadable-in-kibana/303196 "2022-04-27T01:56:55Z")
Hi, I am using filebeat version 7.17.3 running on Ubuntu 18.04.6 LTS to ship docker logs using filebeat installation on the host OS. I am not using a filebeat container. I noticed a couple of things that I would like t… --- ## [After upgrading to 7.17.3 or 8.1.3 from 7.16.3 I get cannot get crawler to start with logs](https://discuss.elastic.co/t/after-upgrading-to-7-17-3-or-8-1-3-from-7-16-3-i-get-cannot-get-crawler-to-start-with-logs/303332)
**Author:** [@Tony\_Powell](https://discuss.elastic.co/u/Tony_Powell)\ **Replies:** 1\ **Last updated:** [April 27, 2022, 12:52am UTC](https://discuss.elastic.co/t/after-upgrading-to-7-17-3-or-8-1-3-from-7-16-3-i-get-cannot-get-crawler-to-start-with-logs/303332 "2022-04-27T00:52:39Z")
in my helm values file I have the following and I was going to comment out the second log to see if it makes a difference processors: # Add Kubernetes metadata | Filebeat Reference \[7.7\] | Elastic - add\_kubernetes\_met… --- ## [Unable to start filebeat service](https://discuss.elastic.co/t/unable-to-start-filebeat-service/303089)
**Author:** [@Bhagavat\_Bhise](https://discuss.elastic.co/u/Bhagavat_Bhise)\ **Replies:** 2\ **Last updated:** [April 26, 2022, 2:36pm UTC](https://discuss.elastic.co/t/unable-to-start-filebeat-service/303089 "2022-04-26T14:36:56Z")
Hi Team, I am facing an issue while starting filebeat service, getting below error. Would you please help me on this? Please find below details for your help. Apr 24 08:45:55 elku systemd\[1\]: filebeat.service: Main p… --- ## [Reduce the metadata generated by filebeat while pulling logs in Kubernetes](https://discuss.elastic.co/t/reduce-the-metadata-generated-by-filebeat-while-pulling-logs-in-kubernetes/303267)
**Author:** [@prity-k](https://discuss.elastic.co/u/prity-k)\ **Replies:** 0\ **Last updated:** [April 26, 2022, 10:53am UTC](https://discuss.elastic.co/t/reduce-the-metadata-generated-by-filebeat-while-pulling-logs-in-kubernetes/303267 "2022-04-26T10:53:23Z")
I am using filebeat(v 8.1.0) to pull logs from microservices deployed in kubernetes env. Right now, there is a huge amount of meta data generated by filebeat apart from the actual log message. Is there a way cut it sort … --- ## [\[BUG\] filebeat becomes totally unreliable with --once option](https://discuss.elastic.co/t/bug-filebeat-becomes-totally-unreliable-with-once-option/303183)
**Author:** [@kRs](https://discuss.elastic.co/u/kRs)\ **Replies:** 5\ **Last updated:** [April 26, 2022, 10:10am UTC](https://discuss.elastic.co/t/bug-filebeat-becomes-totally-unreliable-with-once-option/303183 "2022-04-26T10:10:53Z")
Hi, by using a config as simple as this one: filebeat.inputs: # filestream is an input for collecting log messages from files. - type: filestream id: "test" enabled: true paths: - /home/krs/elasticsearch/\*.log… --- ## [How many metricbeat and filebeat instances should I have in a docker swarm?](https://discuss.elastic.co/t/how-many-metricbeat-and-filebeat-instances-should-i-have-in-a-docker-swarm/302694)
**Author:** [@UchihaYuki](https://discuss.elastic.co/u/UchihaYuki)\ **Replies:** 4\ **Last updated:** [April 26, 2022, 9:46am UTC](https://discuss.elastic.co/t/how-many-metricbeat-and-filebeat-instances-should-i-have-in-a-docker-swarm/302694 "2022-04-26T09:46:17Z")
I'm still pretty new to elastic stack. And I'm come from prometheus and grafana, and I'd like to substitute them for metricbeat, filebeat and kibana. As a newbie, I always wonder, what is the best practice to deploy fil… --- ## [Can I ingest specific log files using winlogbeat?](https://discuss.elastic.co/t/can-i-ingest-specific-log-files-using-winlogbeat/303124)
**Author:** [@rrrrrrrrrrr](https://discuss.elastic.co/u/rrrrrrrrrrr)\ **Replies:** 4\ **Last updated:** [April 26, 2022, 9:31am UTC](https://discuss.elastic.co/t/can-i-ingest-specific-log-files-using-winlogbeat/303124 "2022-04-26T09:31:02Z")
Hello, I would just like to ask if I can ingest a specific logfile using Winlogbeat? If yes, can you provide a structure or link how to do it since I tried researching and no luck finding an answer. Right now here's m… --- ## [Metricbeat 8.1.1 windows service does not startup](https://discuss.elastic.co/t/metricbeat-8-1-1-windows-service-does-not-startup/302816)
**Author:** [@tim135](https://discuss.elastic.co/u/tim135)\ **Replies:** 1\ **Last updated:** [April 26, 2022, 1:36am UTC](https://discuss.elastic.co/t/metricbeat-8-1-1-windows-service-does-not-startup/302816 "2022-04-26T01:36:03Z")
Hi, I'm trying to deploy metrics beat agent 8.1.1 using ansible playbook on our servers. But the service "metricbeat" does not start up. --- ## [Need to send complete xml file from filebeat to logstash in message field](https://discuss.elastic.co/t/need-to-send-complete-xml-file-from-filebeat-to-logstash-in-message-field/302844)
**Author:** [@Rohit\_Goel1](https://discuss.elastic.co/u/Rohit_Goel1)\ **Replies:** 3\ **Last updated:** [April 25, 2022, 11:55am UTC](https://discuss.elastic.co/t/need-to-send-complete-xml-file-from-filebeat-to-logstash-in-message-field/302844 "2022-04-25T11:55:45Z")
Hi , I am using below filebeat config yaml to send below xml to logstash for filteration. but it is not sending complete xml file. and sometimes the xml being sent is not in correct format and tags are getting closed ea… --- ## [Filestream input ID without ID might lead to data duplication](https://discuss.elastic.co/t/filestream-input-id-without-id-might-lead-to-data-duplication/302471)
**Author:** [@jean.bissonnette](https://discuss.elastic.co/u/jean.bissonnette)\ **Replies:** 3\ **Last updated:** [April 25, 2022, 10:53am UTC](https://discuss.elastic.co/t/filestream-input-id-without-id-might-lead-to-data-duplication/302471 "2022-04-25T10:53:49Z")
Filebeat 8.1.2 is installed on a windows server with web applications generating log files and using IIS. I have this error when I start filebeat via windows service. {"log.level":"error","@timestamp":"2022-04-14T15:21… --- ## [Filebeat: docker autodiscover problem](https://discuss.elastic.co/t/filebeat-docker-autodiscover-problem/301452)
**Author:** [@111207](https://discuss.elastic.co/u/111207)\ **Replies:** 3\ **Last updated:** [April 25, 2022, 6:08am UTC](https://discuss.elastic.co/t/filebeat-docker-autodiscover-problem/301452 "2022-04-25T06:08:05Z")
I need to using filebeat auto discover to input the docker container log My container log name pantsel/konga This is my config in filebeat.yml #==============Using Autodiscover======================= filebeat.autodisc… --- ## [What is the correct workflow for standalone elastic agent on docker swarm?](https://discuss.elastic.co/t/what-is-the-correct-workflow-for-standalone-elastic-agent-on-docker-swarm/303093)
**Author:** [@UchihaYuki](https://discuss.elastic.co/u/UchihaYuki)\ **Replies:** 0\ **Last updated:** [April 24, 2022, 10:08am UTC](https://discuss.elastic.co/t/what-is-the-correct-workflow-for-standalone-elastic-agent-on-docker-swarm/303093 "2022-04-24T10:08:51Z")
I don't use fleet server, because I hope everything is up and running in production environment just as what I have configured in development environment. I don't think fleet server can do this. It seems I have to config… --- ## [When the output target stops for a period of time and then resumes, filebeat is in a blocking state](https://discuss.elastic.co/t/when-the-output-target-stops-for-a-period-of-time-and-then-resumes-filebeat-is-in-a-blocking-state/303086)
**Author:** [@lhb6540](https://discuss.elastic.co/u/lhb6540)\ **Replies:** 0\ **Last updated:** [April 24, 2022, 7:51am UTC](https://discuss.elastic.co/t/when-the-output-target-stops-for-a-period-of-time-and-then-resumes-filebeat-is-in-a-blocking-state/303086 "2022-04-24T07:51:34Z")
version: 7.16 os: centos 7.6 install: rpm Steps to Reproduce: When the input of filebeat is an http endpoint, and the output is configured as a logstash. filebeat has a large qps, such as 60/s, you can use a script … --- ## [Autodiscover not working on docker swarm](https://discuss.elastic.co/t/autodiscover-not-working-on-docker-swarm/303082)
**Author:** [@UchihaYuki](https://discuss.elastic.co/u/UchihaYuki)\ **Replies:** 0\ **Last updated:** [April 24, 2022, 6:09am UTC](https://discuss.elastic.co/t/autodiscover-not-working-on-docker-swarm/303082 "2022-04-24T06:09:26Z")
If I config directly in metricbeat.yml: - module: traefik metricsets: - health hosts: \["http://traefik:8080"\] period: 10s enabled: true It will work. But If I use autodiscover, by configuring m… --- ## [Which volumes of beats should I mount under docker swarm?](https://discuss.elastic.co/t/which-volumes-of-beats-should-i-mount-under-docker-swarm/302714)
**Author:** [@UchihaYuki](https://discuss.elastic.co/u/UchihaYuki)\ **Replies:** 2\ **Last updated:** [April 23, 2022, 2:54pm UTC](https://discuss.elastic.co/t/which-volumes-of-beats-should-i-mount-under-docker-swarm/302714 "2022-04-23T14:54:05Z")
I have a custom image for filebeat and metricbeat. So every time when they restart, filebeat setup and metricbeat setup will run before filebeat and metricbeat. I wonder which folders I should mount under a named volumn… --- ## [Harvester\_buffer\_size - how to determine most efficient value?](https://discuss.elastic.co/t/harvester-buffer-size-how-to-determine-most-efficient-value/303037)
**Author:** [@mikewillis](https://discuss.elastic.co/u/mikewillis)\ **Replies:** 0\ **Last updated:** [April 22, 2022, 1:02pm UTC](https://discuss.elastic.co/t/harvester-buffer-size-how-to-determine-most-efficient-value/303037 "2022-04-22T13:02:21Z")
The documentation says what the harvest\_buffer\_size setting of the log input is for, but nothing about why one might want to change it. Log input | Filebeat Reference \[7.17\] | Elastic I asked about this in another post … --- ## [Office 365 filebeat error](https://discuss.elastic.co/t/office-365-filebeat-error/303026)
**Author:** [@ima](https://discuss.elastic.co/u/ima)\ **Replies:** 0\ **Last updated:** [April 22, 2022, 11:58am UTC](https://discuss.elastic.co/t/office-365-filebeat-error/303026 "2022-04-22T11:58:40Z")
Hello, we want to integrate office 365 with filebeat, we have activated the module and filled the config file as shown audit: enabled: true var.application\_id: "\*\*\*\*\* " var.tenants: \*\*- id: "" name: " .onmicrosoft.… --- ## [Elastic-agent in docker - host metadata](https://discuss.elastic.co/t/elastic-agent-in-docker-host-metadata/303000)
**Author:** [@mkf1](https://discuss.elastic.co/u/mkf1)\ **Replies:** 0\ **Last updated:** [April 22, 2022, 9:04am UTC](https://discuss.elastic.co/t/elastic-agent-in-docker-host-metadata/303000 "2022-04-22T09:04:21Z")
Hi, When using elastic-agent in Docker is it possible to override/manually set the host metadata (host.ip, host.mac etc..) to the actual host? When elastic-agent is in a container it select the container ip and containe… --- ## [How to check logs were sending to elasticsearch](https://discuss.elastic.co/t/how-to-check-logs-were-sending-to-elasticsearch/302799)
**Author:** [@Hoa\_Nguy\_n\_Van](https://discuss.elastic.co/u/Hoa_Nguy_n_Van)\ **Replies:** 11\ **Last updated:** [April 22, 2022, 8:31am UTC](https://discuss.elastic.co/t/how-to-check-logs-were-sending-to-elasticsearch/302799 "2022-04-22T08:31:28Z")
I deployment my EFK stack on version 7.4.0. When deploying filebeat, logs from filebeat do not send to Elasticsearch. My configuration Filebeat on k8s: Logs from Filebeat: Help me for this issue. Thanks ! … --- ## [Harvest only till last but one line?](https://discuss.elastic.co/t/harvest-only-till-last-but-one-line/302988)
**Author:** [@gbml](https://discuss.elastic.co/u/gbml)\ **Replies:** 0\ **Last updated:** [April 22, 2022, 7:15am UTC](https://discuss.elastic.co/t/harvest-only-till-last-but-one-line/302988 "2022-04-22T07:15:59Z")
Hello. Im trying to ship logs via filebeat 8.1.2 I have an app that generate text log, but in strange manner: After first event log file look like this: Line 1\\n (space\_symgol\_1)(space\_symgol\_2)...(space\_symgol\_1\_000… --- ## [FIlebeat\[8.1.2\] - Juniper module not able to Setup pipeline](https://discuss.elastic.co/t/filebeat-8-1-2-juniper-module-not-able-to-setup-pipeline/302986)
**Author:** [@Azeem\_Ismail](https://discuss.elastic.co/u/Azeem_Ismail)\ **Replies:** 0\ **Last updated:** [April 22, 2022, 6:54am UTC](https://discuss.elastic.co/t/filebeat-8-1-2-juniper-module-not-able-to-setup-pipeline/302986 "2022-04-22T06:54:07Z")
Hello, After installing filebeat 8.1.3 as an upgrade, Juniper module is not able to setup pipelines. The error received was "Exiting: module juniper is configured but has no enabled filesets" The command ran to setup … --- ## [Redis: Error fetching data for metricset i/o timeout error](https://discuss.elastic.co/t/redis-error-fetching-data-for-metricset-i-o-timeout-error/302813)
**Author:** [@Duane\_DSouza](https://discuss.elastic.co/u/Duane_DSouza)\ **Replies:** 2\ **Last updated:** [April 21, 2022, 2:49pm UTC](https://discuss.elastic.co/t/redis-error-fetching-data-for-metricset-i-o-timeout-error/302813 "2022-04-21T14:49:49Z")
Hi , I am trying to use metricbeat (version 8.1.2) module for Redis. Redis cluster is managed app.redislabs.com Also tried with Redis service for the Google Cloud Platform. Here is config: metricsets: \["info", "ke… --- ## [Multiple hostname (DNS) output for beats?](https://discuss.elastic.co/t/multiple-hostname-dns-output-for-beats/302964)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 0\ **Last updated:** [April 21, 2022, 8:00pm UTC](https://discuss.elastic.co/t/multiple-hostname-dns-output-for-beats/302964 "2022-04-21T20:00:53Z")
Hello, Is it possible to use multiple FQDN's as part of output to logstash via beats? My usecase: I want to send a publically routable logging destination - logging.maniarfamily.com which will resolve to a public IP … --- ## [Filebeat o365 error](https://discuss.elastic.co/t/filebeat-o365-error/302936)
**Author:** [@ima](https://discuss.elastic.co/u/ima)\ **Replies:** 0\ **Last updated:** [April 21, 2022, 3:30pm UTC](https://discuss.elastic.co/t/filebeat-o365-error/302936 "2022-04-21T15:30:16Z")
Hello, we want to integrate office 365 with filebeat, we have activated the module and filled the config file as shown audit: enabled: true var.application\_id: "\*\*\*\*\*" var.tenants: - id: "\*\*\*\*\*\*\*" name: ".onmicros… --- ## [Metricbeat autodiscovery: use 'unique' with Prometheus exporters](https://discuss.elastic.co/t/metricbeat-autodiscovery-use-unique-with-prometheus-exporters/302707)
**Author:** [@Dali\_Ben\_amor](https://discuss.elastic.co/u/Dali_Ben_amor)\ **Replies:** 5\ **Last updated:** [April 21, 2022, 9:34am UTC](https://discuss.elastic.co/t/metricbeat-autodiscovery-use-unique-with-prometheus-exporters/302707 "2022-04-21T09:34:35Z")
Metricbeat 7.16.2 Hi, I'm trying to scrape prometheus metrics of multiple pods on different nodes from the same metricbeat pod. Actually metricbeat is deployed as a daemonset and prometheus collection is done on node … --- ## [AWS RDS module does not poll metrics regularly](https://discuss.elastic.co/t/aws-rds-module-does-not-poll-metrics-regularly/302393)
**Author:** [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)\ **Replies:** 3\ **Last updated:** [April 20, 2022, 5:34pm UTC](https://discuss.elastic.co/t/aws-rds-module-does-not-poll-metrics-regularly/302393 "2022-04-20T17:34:36Z")
Hello Everyone, I have recently upgraded the Elastic Stack from 7.17.1 to 8.1.2. All the metrics are coming in fine except for AWS RDS which is coming very rarely even though the period is set to 60s. The IAM user has … --- ## [Missing event field from filebeat via docker autodiscovery](https://discuss.elastic.co/t/missing-event-field-from-filebeat-via-docker-autodiscovery/302843)
**Author:** [@galkinrost](https://discuss.elastic.co/u/galkinrost)\ **Replies:** 0\ **Last updated:** [April 20, 2022, 5:06pm UTC](https://discuss.elastic.co/t/missing-event-field-from-filebeat-via-docker-autodiscovery/302843 "2022-04-20T17:06:46Z")
Hi! I'm trying to set up filebeat with docker autodiscovery. But for some reason the event field is missing in documents. Here the config: filebeat.config: modules: path: ${path.config}/modules.d/\*.yml reloa… --- ## [Exclude\_files param not working](https://discuss.elastic.co/t/exclude-files-param-not-working/302803)
**Author:** [@yyovchev](https://discuss.elastic.co/u/yyovchev)\ **Replies:** 1\ **Last updated:** [April 20, 2022, 4:05pm UTC](https://discuss.elastic.co/t/exclude-files-param-not-working/302803 "2022-04-20T16:05:15Z")
Hello, I try to exclude only one log file from filebeat, but the records are still sent to Elasticsearch. Here is my /etc/filebeat/modules.d/nginx.yml file: # Module: nginx # Docs: https://www.elastic.co/guide/en/beats… --- ## [Event.module Field Not Added by Winlogbeat Module (\>= v8.0.0)](https://discuss.elastic.co/t/event-module-field-not-added-by-winlogbeat-module-v8-0-0/302595)
**Author:** [@inf](https://discuss.elastic.co/u/inf)\ **Replies:** 3\ **Last updated:** [April 20, 2022, 7:27am UTC](https://discuss.elastic.co/t/event-module-field-not-added-by-winlogbeat-module-v8-0-0/302595 "2022-04-20T07:27:43Z")
Hi there, As far as I understand, event.module is not being added to each document originating from Winlogbeat (starting from v8.0.0) that uses Winlogbeat modules. This field was previously seen from v7.4.0 onwards, wh… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=93) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=95)