# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=343 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 344 --- ## [Kubernetes Metadata Not Being Add Via Filebeat](https://discuss.elastic.co/t/kubernetes-metadata-not-being-add-via-filebeat/187657)
**Author:** [@tnemu](https://discuss.elastic.co/u/tnemu)\ **Replies:** 0\ **Last updated:** [June 26, 2019, 7:17pm UTC](https://discuss.elastic.co/t/kubernetes-metadata-not-being-add-via-filebeat/187657 "2019-06-26T19:17:00Z")
We recently moved to Filebeat 7.x however, we no longer get kubernetes metadata added to our events. We are using Filebeat 7.1.1 and Filebeat 7.2.0. This worked just fine in Filebeat 6.3.2. Also using Kubernetes 1.12 H… --- ## [Unable to decode gzip compressed http response from Packetbeat file output](https://discuss.elastic.co/t/unable-to-decode-gzip-compressed-http-response-from-packetbeat-file-output/187382)
**Author:** [@Mark\_Ryder](https://discuss.elastic.co/u/Mark_Ryder)\ **Replies:** 1\ **Last updated:** [June 26, 2019, 7:15pm UTC](https://discuss.elastic.co/t/unable-to-decode-gzip-compressed-http-response-from-packetbeat-file-output/187382 "2019-06-26T19:15:27Z")
We are currently trying to capture SOAP request and response body content using Packetbeat 7.1.1 (As shown in in screenshot - PacketbeatOutputConfig). Packetbeat is configured to output to file (as shown in screenshot up… --- ## [Empty @metadata fields when upgrading to 7.1.x](https://discuss.elastic.co/t/empty-metadata-fields-when-upgrading-to-7-1-x/187081)
**Author:** [@boernd](https://discuss.elastic.co/u/boernd)\ **Replies:** 2\ **Last updated:** [June 26, 2019, 3:48pm UTC](https://discuss.elastic.co/t/empty-metadata-fields-when-upgrading-to-7-1-x/187081 "2019-06-26T15:48:51Z")
Hello, I'm trying to upgrade my custom beat to 7.1.x (from 6.x) and encounter the issue that some fields of the @metadata json are empty: "@metadata": { "beat": "", "type": "\_doc", "version": "" }, I also tested c… --- ## [Filebeat.autodiscover](https://discuss.elastic.co/t/filebeat-autodiscover/187592)
**Author:** [@sszabo](https://discuss.elastic.co/u/sszabo)\ **Replies:** 1\ **Last updated:** [June 26, 2019, 2:39pm UTC](https://discuss.elastic.co/t/filebeat-autodiscover/187592 "2019-06-26T14:39:16Z")
If we have a single filebeat pod running on a kubernetes worker node with two inputs both using the same container id list will this cause a problem? filebeat.autodiscover: providers: - type: kubernetes … --- ## [How to make a single watcher monitor all the services](https://discuss.elastic.co/t/how-to-make-a-single-watcher-monitor-all-the-services/187048)
**Author:** [@Naveenraj](https://discuss.elastic.co/u/Naveenraj)\ **Replies:** 6\ **Last updated:** [June 26, 2019, 2:35pm UTC](https://discuss.elastic.co/t/how-to-make-a-single-watcher-monitor-all-the-services/187048 "2019-06-26T14:35:30Z")
Hi, I am new to watcher, sorry if it sounds silly. Can you please help to make a single watcher in ELK to monitor all of my micro services(i have 7), currently i can get alerting, with one watcher per service. Followi… --- ## [Monitoring Spark and Kafka performance with Metricbeat](https://discuss.elastic.co/t/monitoring-spark-and-kafka-performance-with-metricbeat/187588)
**Author:** [@cjb312](https://discuss.elastic.co/u/cjb312)\ **Replies:** 0\ **Last updated:** [June 26, 2019, 2:03pm UTC](https://discuss.elastic.co/t/monitoring-spark-and-kafka-performance-with-metricbeat/187588 "2019-06-26T14:03:55Z")
I'm looking to monitor performance of both Kafka and Spark using Metricbeat. For example, how long it takes to send a message with Kafka and how long Spark takes to complete a job. Whether you have direct advice/examp… --- ## [Functionbeat cloudwatch S3 AccessDenied](https://discuss.elastic.co/t/functionbeat-cloudwatch-s3-accessdenied/187432)
**Author:** [@Nivead\_Transposit](https://discuss.elastic.co/u/Nivead_Transposit)\ **Replies:** 4\ **Last updated:** [June 26, 2019, 1:56pm UTC](https://discuss.elastic.co/t/functionbeat-cloudwatch-s3-accessdenied/187432 "2019-06-26T13:56:37Z")
Following the documents, I am trying to configure functionbeat to ingest cloudwatch logs to elastic cloud deployment. I have already given 'AmazonS3FullAccess' policy to the IAM user I use for this operation. I just tes… --- ## [Kubernetes Autodiscover Setup & Logging](https://discuss.elastic.co/t/kubernetes-autodiscover-setup-logging/187582)
**Author:** [@remmeier](https://discuss.elastic.co/u/remmeier)\ **Replies:** 0\ **Last updated:** [June 26, 2019, 1:44pm UTC](https://discuss.elastic.co/t/kubernetes-autodiscover-setup-logging/187582 "2019-06-26T13:44:53Z")
Hi, I'm attempting to get the Kubernetes auto-discovery working, but failed so far. My configuration looks like: metricbeat.config: modules: path: ${path.config}/modules.d/\*.yml reload.enabled: false … --- ## [Error with filebeat log type that was working on different machine](https://discuss.elastic.co/t/error-with-filebeat-log-type-that-was-working-on-different-machine/187435)
**Author:** [@aviationfan](https://discuss.elastic.co/u/aviationfan)\ **Replies:** 2\ **Last updated:** [June 26, 2019, 1:01pm UTC](https://discuss.elastic.co/t/error-with-filebeat-log-type-that-was-working-on-different-machine/187435 "2019-06-26T13:01:29Z")
I was working with a Filebeat setup on my laptop and decided to move it to my server for more permanent use. I keep getting this error: Exiting: 1 error: setting 'filebeat.prospectors' has been removed The contents of … --- ## [Metricbeat 7.0.1 system.filesystem.ignore\_types not working correctly](https://discuss.elastic.co/t/metricbeat-7-0-1-system-filesystem-ignore-types-not-working-correctly/187572)
**Author:** [@miksonx](https://discuss.elastic.co/u/miksonx)\ **Replies:** 0\ **Last updated:** [June 26, 2019, 1:01pm UTC](https://discuss.elastic.co/t/metricbeat-7-0-1-system-filesystem-ignore-types-not-working-correctly/187572 "2019-06-26T13:01:23Z")
Hi, I have setup metricbeats to elastic to collect metrics from host filesystem running under openshift kubernetes pod Seems that the filesystem.ignore\_types only collect random or latest in the list of the file system … --- ## [Unable to use own index name](https://discuss.elastic.co/t/unable-to-use-own-index-name/187510)
**Author:** [@ssharaf](https://discuss.elastic.co/u/ssharaf)\ **Replies:** 2\ **Last updated:** [June 26, 2019, 8:04am UTC](https://discuss.elastic.co/t/unable-to-use-own-index-name/187510 "2019-06-26T08:04:05Z")
Greetings, I started experimenting Elasticsearch with importing logs from different servers and docker containers using Filebeat. I was able to use KQL and build custom Kibana dashboards which is very good to better mon… --- ## [Filebeat 7.2.0 - Cisco ASA module not parsing IPv6 correctly](https://discuss.elastic.co/t/filebeat-7-2-0-cisco-asa-module-not-parsing-ipv6-correctly/187509)
**Author:** [@elastic22](https://discuss.elastic.co/u/elastic22)\ **Replies:** 0\ **Last updated:** [June 26, 2019, 7:47am UTC](https://discuss.elastic.co/t/filebeat-7-2-0-cisco-asa-module-not-parsing-ipv6-correctly/187509 "2019-06-26T07:47:59Z")
Hello guys I was excited to hear that there is now a built-in module for Cisco ASA in filebeat. I quickly tried it in our test environment and saw the following issue: The ASA will log HTTP requests that it is able to … --- ## [Metribeat 7.1.1 not logging to files, only syslog always](https://discuss.elastic.co/t/metribeat-7-1-1-not-logging-to-files-only-syslog-always/187088)
**Author:** [@rebirther](https://discuss.elastic.co/u/rebirther)\ **Replies:** 1\ **Last updated:** [June 26, 2019, 7:17am UTC](https://discuss.elastic.co/t/metribeat-7-1-1-not-logging-to-files-only-syslog-always/187088 "2019-06-26T07:17:47Z")
Hello Metricbeat does not stop sending logs to syslog. Even after parameter in metricbeat.yml: logging.to\_syslog: false This happens if you run metricbeat (ubuntu 16.04): service metricbeat start It helps only run … --- ## [Enroll filebeat in Kubernetes, monitor in Kibana when output is kafka possible?](https://discuss.elastic.co/t/enroll-filebeat-in-kubernetes-monitor-in-kibana-when-output-is-kafka-possible/185892)
**Author:** [@alonchis](https://discuss.elastic.co/u/alonchis)\ **Replies:** 1\ **Last updated:** [June 25, 2019, 8:57pm UTC](https://discuss.elastic.co/t/enroll-filebeat-in-kubernetes-monitor-in-kibana-when-output-is-kafka-possible/185892 "2019-06-25T20:57:53Z")
I want to enroll filebeat running in a kubernetes cluster outputting to kafka. In kibana it says to run that sudo filebeat enroll \[API\_KEY\] command but since this is running kubernetes, it gets kinda tricky. Is there a… --- ## [Are re-index and delete operations possible with libbeat?](https://discuss.elastic.co/t/are-re-index-and-delete-operations-possible-with-libbeat/184856)
**Author:** [@whatgeorgemade](https://discuss.elastic.co/u/whatgeorgemade)\ **Replies:** 3\ **Last updated:** [June 25, 2019, 8:16pm UTC](https://discuss.elastic.co/t/are-re-index-and-delete-operations-possible-with-libbeat/184856 "2019-06-25T20:16:18Z")
I've written a Beat and it's working like a champ for our time-series data. However, we also have management data we'd like to index and I've hit a problem. The management has timestamps and ID values for every document… --- ## [Functionbeat Sending to Wrong Index](https://discuss.elastic.co/t/functionbeat-sending-to-wrong-index/187415)
**Author:** [@jeffreygatsby](https://discuss.elastic.co/u/jeffreygatsby)\ **Replies:** 0\ **Last updated:** [June 25, 2019, 6:55pm UTC](https://discuss.elastic.co/t/functionbeat-sending-to-wrong-index/187415 "2019-06-25T18:55:19Z")
I'm attempting to change the index that FunctionBeat sends to. Here is my configuration: functionbeat.provider.aws.deploy\_bucket: "gatsby-deploy" functionbeat.provider.aws.functions: - name: EcsStagingLogs enable… --- ## [Running filebeat as a docker image](https://discuss.elastic.co/t/running-filebeat-as-a-docker-image/187402)
**Author:** [@exocore123](https://discuss.elastic.co/u/exocore123)\ **Replies:** 1\ **Last updated:** [June 25, 2019, 6:42pm UTC](https://discuss.elastic.co/t/running-filebeat-as-a-docker-image/187402 "2019-06-25T18:42:28Z")
Hello all, Currently I am trying to implement filebeat into an EC2 instance to do logshipping from this ec2 instance to another ec2 (that is the kafka's). ContainerDefinitions: - Name: !Ref service … --- ## [Modules dont Log?](https://discuss.elastic.co/t/modules-dont-log/187155)
**Author:** [@shinerrs](https://discuss.elastic.co/u/shinerrs)\ **Replies:** 6\ **Last updated:** [June 25, 2019, 5:31pm UTC](https://discuss.elastic.co/t/modules-dont-log/187155 "2019-06-25T17:31:50Z")
Hello, I have been trying to deal with the horrible event managment of metricbeat. I find it a terrible design that the main configuration is logged but when it comes to modules you need to run a special event run to se… --- ## [No matching indices found: No indices match pattern "winlogbeat-\*" while running winlogbeat](https://discuss.elastic.co/t/no-matching-indices-found-no-indices-match-pattern-winlogbeat-while-running-winlogbeat/187244)
**Author:** [@Vishnu\_mk](https://discuss.elastic.co/u/Vishnu_mk)\ **Replies:** 1\ **Last updated:** [June 25, 2019, 4:47pm UTC](https://discuss.elastic.co/t/no-matching-indices-found-no-indices-match-pattern-winlogbeat-while-running-winlogbeat/187244 "2019-06-25T16:47:54Z")
This is my winlogbeat configuration: ###################### Winlogbeat Configuration Example ########################## # This file is an example configuration file highlighting only the most common # options. The winl… --- ## [Custom beat package failure](https://discuss.elastic.co/t/custom-beat-package-failure/185939)
**Author:** [@pawankt](https://discuss.elastic.co/u/pawankt)\ **Replies:** 1\ **Last updated:** [June 25, 2019, 3:51pm UTC](https://discuss.elastic.co/t/custom-beat-package-failure/185939 "2019-06-25T15:51:28Z")
Hi, I was able to make build for my custom beat. How ever make package failing to create pkg. When i run make package it just rotating to download make\_output\_file.go, never proceeding to next step. Any help much appreci… --- ## [Capturing USB and IRP traffic](https://discuss.elastic.co/t/capturing-usb-and-irp-traffic/186162)
**Author:** [@El\_Mahdi\_El\_Korri](https://discuss.elastic.co/u/El_Mahdi_El_Korri)\ **Replies:** 1\ **Last updated:** [June 25, 2019, 2:32pm UTC](https://discuss.elastic.co/t/capturing-usb-and-irp-traffic/186162 "2019-06-25T14:32:11Z")
Hello everyone, Is there any way to capture USB and IRP (IO Request Packet) with packetbeat? I am looking forward to hearing your feedback. Best regards, --- ## [Metricbeat Kibana Dashboard for Postgresql](https://discuss.elastic.co/t/metricbeat-kibana-dashboard-for-postgresql/183544)
**Author:** [@RockD](https://discuss.elastic.co/u/RockD)\ **Replies:** 10\ **Last updated:** [June 25, 2019, 1:04pm UTC](https://discuss.elastic.co/t/metricbeat-kibana-dashboard-for-postgresql/183544 "2019-06-25T13:04:24Z")
Hi I have installed metricbeat in a machine with a postgresql service on it, so I enabled the postgresql module of metricbeat. I loaded the dashboards into kibana and search for a postgresql dashboard and doesn't exist… --- ## [Exact mongo query not coming in packet beats](https://discuss.elastic.co/t/exact-mongo-query-not-coming-in-packet-beats/187304)
**Author:** [@Abhimanyu\_Nagrath](https://discuss.elastic.co/u/Abhimanyu_Nagrath)\ **Replies:** 0\ **Last updated:** [June 25, 2019, 10:41am UTC](https://discuss.elastic.co/t/exact-mongo-query-not-coming-in-packet-beats/187304 "2019-06-25T10:41:33Z")
Hi I have a mongo 4.0 (config(27018), shard(27021) and monogos(27017) running on the same machine) with SSL enabled. I have enabled packetbeat 6.7 on the server . Rest all stats look fine . But in top 10 slow queries I a… --- ## [Filebeat -\> logstash -\> SIEM](https://discuss.elastic.co/t/filebeat-logstash-siem/187260)
**Author:** [@Vladx](https://discuss.elastic.co/u/Vladx)\ **Replies:** 0\ **Last updated:** [June 25, 2019, 8:24am UTC](https://discuss.elastic.co/t/filebeat-logstash-siem/187260 "2019-06-25T08:24:05Z")
Hi, I'm trying to put together a pilot with ELK plus Qradar. The ELK part works like a charm, but I have some issues on Qradar side. This is heavily relies on proper syslog fields, but unfortunately when I forward logs … --- ## [XML log file with multiple opening/closing tags](https://discuss.elastic.co/t/xml-log-file-with-multiple-opening-closing-tags/187089)
**Author:** [@moltubakk](https://discuss.elastic.co/u/moltubakk)\ **Replies:** 1\ **Last updated:** [June 25, 2019, 7:51am UTC](https://discuss.elastic.co/t/xml-log-file-with-multiple-opening-closing-tags/187089 "2019-06-25T07:51:48Z")
I'm trying to use Filebeat to ship a number of xml log files that follow a quite simple request/response pattern. I thought this would be easy, but as a newcomer to the ELK stack I really need some help! Except from the… --- ## [Ignore lines which are not JSON in log file using Filebeat](https://discuss.elastic.co/t/ignore-lines-which-are-not-json-in-log-file-using-filebeat/187154)
**Author:** [@Jitendra\_Kumhar](https://discuss.elastic.co/u/Jitendra_Kumhar)\ **Replies:** 3\ **Last updated:** [June 25, 2019, 7:48am UTC](https://discuss.elastic.co/t/ignore-lines-which-are-not-json-in-log-file-using-filebeat/187154 "2019-06-25T07:48:57Z")
Sample log file { "name": "Meowsy", "species" : "cat", "foods": { "likes": \["tuna", "catnip"\], "dislikes": \["ham", "zucchini"\] } } another line next line So I want filebeat to ignore 2nd and 3rd line and send only 1s… --- ## [How to parse nginx log using filebeat](https://discuss.elastic.co/t/how-to-parse-nginx-log-using-filebeat/186266)
**Author:** [@prashantgcloud](https://discuss.elastic.co/u/prashantgcloud)\ **Replies:** 2\ **Last updated:** [June 25, 2019, 6:29am UTC](https://discuss.elastic.co/t/how-to-parse-nginx-log-using-filebeat/186266 "2019-06-25T06:29:11Z")
I have setup Elasticsearch and kibana using AWS Elastic search service so can't install below plugin : bin/elasticsearch-plugin install ingest-geoip bin/elasticsearch-plugin install ingest-user-agent I have installed … --- ## [How to parse json.log using filebeat 6.2](https://discuss.elastic.co/t/how-to-parse-json-log-using-filebeat-6-2/186260)
**Author:** [@prashantgcloud](https://discuss.elastic.co/u/prashantgcloud)\ **Replies:** 2\ **Last updated:** [June 25, 2019, 6:28am UTC](https://discuss.elastic.co/t/how-to-parse-json-log-using-filebeat-6-2/186260 "2019-06-25T06:28:45Z")
When I'm setting json.keys\_under\_root: false I'm able to push log in json format to elastic search, but when I set it to true it gives below error: Private:file.State{Id:"", Finished:false, Fileinfo:(\*os.fileStat)(0xc42… --- ## [New to beats - A config question](https://discuss.elastic.co/t/new-to-beats-a-config-question/187070)
**Author:** [@Mats](https://discuss.elastic.co/u/Mats)\ **Replies:** 2\ **Last updated:** [June 25, 2019, 6:24am UTC](https://discuss.elastic.co/t/new-to-beats-a-config-question/187070 "2019-06-25T06:24:36Z")
As stated I'm new to beats and are looking at Beats as a replacement for NXlog in the first place. in the documentation for Winlogbeat it says that i Should configure it for Kibana (Configure the Kibana endpoint). I do… --- ## [Harvester started for file but not reading logs](https://discuss.elastic.co/t/harvester-started-for-file-but-not-reading-logs/187062)
**Author:** [@Vaibhav\_Kumar\_Jain](https://discuss.elastic.co/u/Vaibhav_Kumar_Jain)\ **Replies:** 2\ **Last updated:** [June 25, 2019, 5:36am UTC](https://discuss.elastic.co/t/harvester-started-for-file-but-not-reading-logs/187062 "2019-06-25T05:36:47Z")
\--------------------filebeat.log----------------------- 2019-06-24T12:26:31.687+0530 INFO \[monitoring\] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":375},"t… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=342) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=344)