# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=203 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 204 --- ## [Filebeats live shipping of data?](https://discuss.elastic.co/t/filebeats-live-shipping-of-data/253103)
**Author:** [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\ **Replies:** 0\ **Last updated:** [October 23, 2020, 8:48am UTC](https://discuss.elastic.co/t/filebeats-live-shipping-of-data/253103 "2020-10-23T08:48:23Z")
Hi :slight\_smile: How can I create live shipping of data from filebeats to logstash ? Our Filebeats send data just after restart of docker container. A few minutes later well get this logs ,"filebeat":{"harvester":{"o… --- ## [Alerting if a beat is down](https://discuss.elastic.co/t/alerting-if-a-beat-is-down/252308)
**Author:** [@sfenman](https://discuss.elastic.co/u/sfenman)\ **Replies:** 3\ **Last updated:** [October 23, 2020, 6:26am UTC](https://discuss.elastic.co/t/alerting-if-a-beat-is-down/252308 "2020-10-23T06:26:04Z")
Hello, I have many beats running in different hosts. I need to monitor these agents so I' ve enabled the xpack.monitoring and now I can see event rate, fail rate, cpu util etc. in Kibana. Can I Somehow create alerts for… --- ## [Filebeat netflow module inverting source and destination on host behind nat?](https://discuss.elastic.co/t/filebeat-netflow-module-inverting-source-and-destination-on-host-behind-nat/253023)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 1\ **Last updated:** [October 22, 2020, 9:11pm UTC](https://discuss.elastic.co/t/filebeat-netflow-module-inverting-source-and-destination-on-host-behind-nat/253023 "2020-10-22T21:11:26Z")
Hello, recently when I was working on a network problem I discovered the strange behavior of Kibana parsing the NetFlow docs. I was looking for a top communication on the network and I noticed communication between hos… --- ## [Winlogbeat missing mapping fields](https://discuss.elastic.co/t/winlogbeat-missing-mapping-fields/252846)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 18\ **Last updated:** [October 23, 2020, 9:03am UTC](https://discuss.elastic.co/t/winlogbeat-missing-mapping-fields/252846 "2020-10-23T09:03:44Z")
Hi, I have just installed Winlogbeat on one of my Windows 2019 terminal servers. After configurion the .yml file I have entered setup commands: winlogbeat.exe setup --index-management winlogbeat.exe setup --dahsboar… --- ## [Filebeat cannot parse a custom @timestamp string](https://discuss.elastic.co/t/filebeat-cannot-parse-a-custom-timestamp-string/252972)
**Author:** [@mattia\_galati](https://discuss.elastic.co/u/mattia_galati)\ **Replies:** 1\ **Last updated:** [October 23, 2020, 7:31am UTC](https://discuss.elastic.co/t/filebeat-cannot-parse-a-custom-timestamp-string/252972 "2020-10-23T07:31:03Z")
Hello everyone, I tryed to search in several topics but cannot manage to find a solution. I have a Python application that is writing logs and a Logstash process which is picking up that logs to send them to an Elastic… --- ## [Filebeat not reading .csv and .bad files from a particular location](https://discuss.elastic.co/t/filebeat-not-reading-csv-and-bad-files-from-a-particular-location/253091)
**Author:** [@Pavitra\_Poojary](https://discuss.elastic.co/u/Pavitra_Poojary)\ **Replies:** 0\ **Last updated:** [October 23, 2020, 7:19am UTC](https://discuss.elastic.co/t/filebeat-not-reading-csv-and-bad-files-from-a-particular-location/253091 "2020-10-23T07:19:11Z")
Hi , I am trying to read files from the below 2log paths. #=========================== Filebeat inputs ============================= filebeat.inputs: # Each - is an input. Most options can be set at the input level, … --- ## [Heartbeat Crashed on using a third-party module across the month](https://discuss.elastic.co/t/heartbeat-crashed-on-using-a-third-party-module-across-the-month/252918)
**Author:** [@lowry](https://discuss.elastic.co/u/lowry)\ **Replies:** 2\ **Last updated:** [October 23, 2020, 6:07am UTC](https://discuss.elastic.co/t/heartbeat-crashed-on-using-a-third-party-module-across-the-month/252918 "2020-10-23T06:07:39Z")
Hello, Beats friends, We encountered an issue when we using Heartbeat in our environment. After some further digging, we believe it cased by using a third-party module -- https://github.com/gorhill/cronexpr without con… --- ## [\[feature request\] Add URL Parse processors](https://discuss.elastic.co/t/feature-request-add-url-parse-processors/253084)
**Author:** [@OhBonsai](https://discuss.elastic.co/u/OhBonsai)\ **Replies:** 0\ **Last updated:** [October 23, 2020, 5:21am UTC](https://discuss.elastic.co/t/feature-request-add-url-parse-processors/253084 "2020-10-23T05:21:04Z")
Hi All: It's very common url format field in log. urldecode processor have been implements in libbeat. Add a net/url.Parse processor will be helpful :slight\_smile: I have already done this feature? May I open a Pull… --- ## [Filebeat collect log and output to es but occur time parse error](https://discuss.elastic.co/t/filebeat-collect-log-and-output-to-es-but-occur-time-parse-error/252820)
**Author:** [@lucasyu](https://discuss.elastic.co/u/lucasyu)\ **Replies:** 6\ **Last updated:** [October 23, 2020, 1:38am UTC](https://discuss.elastic.co/t/filebeat-collect-log-and-output-to-es-but-occur-time-parse-error/252820 "2020-10-23T01:38:14Z")
Dear all I use filebeat to collect log and parse to json output to es but the "timestamp" field can not parse the value of timestamp is "2020-10-21 17:10:51.963" error description is "failed to parse date field \[202… --- ## [Filebeat connect with Kafka Kerberos(SASL\_SSL) not working](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160)
**Author:** [@chandramouli\_sriniva](https://discuss.elastic.co/u/chandramouli_sriniva)\ **Replies:** 9\ **Last updated:** [October 15, 2020, 1:41pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160 "2020-10-15T13:41:49Z")
Hi - I am using Filebeat 7.9 version. using filebeat, read data from log file and push to kafka topic. our Kafka use SASL\_SSL with Kerberos mechanism. when I set up with below parms, it errors out. kerberos.enabled: "tr… --- ## [MetricBeat error while connecting to Elasticsearch](https://discuss.elastic.co/t/metricbeat-error-while-connecting-to-elasticsearch/252905)
**Author:** [@Sara3](https://discuss.elastic.co/u/Sara3)\ **Replies:** 2\ **Last updated:** [October 22, 2020, 9:29pm UTC](https://discuss.elastic.co/t/metricbeat-error-while-connecting-to-elasticsearch/252905 "2020-10-22T21:29:29Z")
Hi , I am having trouble with metricbeat communicate to OpendistroElasticsearch .Both are oss version.Please help! Error seen is : ERROR \[publisher\_pipeline\_output\] pipeline/output.go:154 Failed t… --- ## [Mentioning path in filebeat.yml](https://discuss.elastic.co/t/mentioning-path-in-filebeat-yml/253018)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 1\ **Last updated:** [October 22, 2020, 3:55pm UTC](https://discuss.elastic.co/t/mentioning-path-in-filebeat-yml/253018 "2020-10-22T15:55:49Z")
Hi All, The name of the log file that I need to mention in filebeat.yml is access-202010210000 I tried putting the following but it does not seem to work. Please help: filebeat.inputs: type: log paths: /opt/table… --- ## [Configure ILM in filebeat](https://discuss.elastic.co/t/configure-ilm-in-filebeat/252954)
**Author:** [@gizem](https://discuss.elastic.co/u/gizem)\ **Replies:** 7\ **Last updated:** [October 22, 2020, 1:16pm UTC](https://discuss.elastic.co/t/configure-ilm-in-filebeat/252954 "2020-10-22T13:16:19Z")
Hello, I was using 7.0.1 version of filebeat before, and my index configuration line this: setup.template.name: "index" setup.template.pattern: "index-\*" output.elasticsearch: index: "index-%{\[fields.type\]:other}-%{… --- ## [Kubernetes Metricbeat](https://discuss.elastic.co/t/kubernetes-metricbeat/252666)
**Author:** [@Anchit\_Aanand](https://discuss.elastic.co/u/Anchit_Aanand)\ **Replies:** 2\ **Last updated:** [October 22, 2020, 1:12pm UTC](https://discuss.elastic.co/t/kubernetes-metricbeat/252666 "2020-10-22T13:12:45Z")
Hi I'm getting below errors while fetching kubernetes metrices with metricbeat. Error fetching data for metricset kubernetes.pod: error doing HTTP request to fetch 'pod' Metricset data: HTTP error 403 in : 403 Forbidden … --- ## [Metricbeat 7.9.2 is shutdown silently](https://discuss.elastic.co/t/metricbeat-7-9-2-is-shutdown-silently/252945)
**Author:** [@Roy\_Zhang](https://discuss.elastic.co/u/Roy_Zhang)\ **Replies:** 2\ **Last updated:** [October 22, 2020, 12:42pm UTC](https://discuss.elastic.co/t/metricbeat-7-9-2-is-shutdown-silently/252945 "2020-10-22T12:42:27Z")
Dear Metricbeat experts, I found metricbeat 7.9.2 was shutdown sliently, I enable debug log, from log, it said service/service.go:56 Received sighup, stopping. I started metricbeat using root user, I search the histor… --- ## [Can't connect to AWS Elasticsearch from EC2](https://discuss.elastic.co/t/cant-connect-to-aws-elasticsearch-from-ec2/252859)
**Author:** [@CatalinM](https://discuss.elastic.co/u/CatalinM)\ **Replies:** 1\ **Last updated:** [October 22, 2020, 11:17am UTC](https://discuss.elastic.co/t/cant-connect-to-aws-elasticsearch-from-ec2/252859 "2020-10-22T11:17:35Z")
I've did a lot of digging and I learned that I need to use an OSS version of Heartbeat to achieve this. As such, I removed the version I had initially and installed an OSS version (initially 7.9.2, but also tried 7.7.0). … --- ## [MSSQL monitoring with Metricbeat?](https://discuss.elastic.co/t/mssql-monitoring-with-metricbeat/252733)
**Author:** [@ManuelF](https://discuss.elastic.co/u/ManuelF)\ **Replies:** 5\ **Last updated:** [October 22, 2020, 10:12am UTC](https://discuss.elastic.co/t/mssql-monitoring-with-metricbeat/252733 "2020-10-22T10:12:59Z")
Hi, Note: - Using ELK 7.9.2 in Debian 10 - MS SQL Server 2016 SP2 Standard in Windows Server 2012 I am trying to monitor my MS SQL Server with Elasticsearch and visualize the data in Kibana. After some research I fou… --- ## [Using secrets in the heartbeat http monitor proxy url](https://discuss.elastic.co/t/using-secrets-in-the-heartbeat-http-monitor-proxy-url/252792)
**Author:** [@halltony](https://discuss.elastic.co/u/halltony)\ **Replies:** 2\ **Last updated:** [October 21, 2020, 1:17pm UTC](https://discuss.elastic.co/t/using-secrets-in-the-heartbeat-http-monitor-proxy-url/252792 "2020-10-21T13:17:44Z")
Our proxy server requires credentials. I don't want these in plain text in the heartbeat configuration and would there like to add them to a keystore and use syntax like: proxy\_url: "http://${USER}:${PASS}@myproxy:80" … --- ## [Filebeat syslog unix socket input doesn't work for Nginx](https://discuss.elastic.co/t/filebeat-syslog-unix-socket-input-doesnt-work-for-nginx/252958)
**Author:** [@zoulja](https://discuss.elastic.co/u/zoulja)\ **Replies:** 0\ **Last updated:** [October 22, 2020, 8:06am UTC](https://discuss.elastic.co/t/filebeat-syslog-unix-socket-input-doesnt-work-for-nginx/252958 "2020-10-22T08:06:08Z")
I'm trying to collect customized Nginx access logs via Filebeat using syslog unix socket type. Filebeat 7.9.2: - input\_type: syslog protocol.unix: path: "/var/log/nginx/nginx.sock" Nginx 1.15.2: access\_log sy… --- ## [Filebeat config using Docker labels as index and pipeline names](https://discuss.elastic.co/t/filebeat-config-using-docker-labels-as-index-and-pipeline-names/252959)
**Author:** [@jbws](https://discuss.elastic.co/u/jbws)\ **Replies:** 0\ **Last updated:** [October 22, 2020, 8:08am UTC](https://discuss.elastic.co/t/filebeat-config-using-docker-labels-as-index-and-pipeline-names/252959 "2020-10-22T08:08:16Z")
I am trying to use docker labels as the index and pipeline names. What is wrong with this config? filebeat.config: modules: path: ${path.config}/modules.d/\*.yml reload.enabled: false filebeat.autodiscover: … --- ## [Create new Filebeat](https://discuss.elastic.co/t/create-new-filebeat/246081)
**Author:** [@saffatechy](https://discuss.elastic.co/u/saffatechy)\ **Replies:** 34\ **Last updated:** [October 22, 2020, 8:01am UTC](https://discuss.elastic.co/t/create-new-filebeat/246081 "2020-10-22T08:01:22Z")
Hi Elastic community I'm trying to build a custom Filebeat module but to me the Doc is a little unspecific: https://www.elastic.co/guide/en/beats/devguide/current/filebeat-modules-devguide.html I've come as far as crea… --- ## [Strange Filebeat alerts processing](https://discuss.elastic.co/t/strange-filebeat-alerts-processing/252345)
**Author:** [@MichaelA](https://discuss.elastic.co/u/MichaelA)\ **Replies:** 16\ **Last updated:** [October 22, 2020, 7:35am UTC](https://discuss.elastic.co/t/strange-filebeat-alerts-processing/252345 "2020-10-22T07:35:13Z")
Hi, dear community! We've faced the following problem in our Elastic stack: Filebeat sends the processed logs - about 100-150 Gb in average per day, but instead of straight line as it always be now we see such peaks (se… --- ## [Filebeat manage multiple write indices](https://discuss.elastic.co/t/filebeat-manage-multiple-write-indices/252760)
**Author:** [@keshara](https://discuss.elastic.co/u/keshara)\ **Replies:** 2\ **Last updated:** [October 22, 2020, 2:24am UTC](https://discuss.elastic.co/t/filebeat-manage-multiple-write-indices/252760 "2020-10-22T02:24:44Z")
According to the link =\> https://www.elastic.co/guide/en/beats/filebeat/6.8/ilm.html#\_advanced\_ilm\_settings I am trying to create multiple indices from filebeat output to elasticsearch. Here is my filebeat.yml. - … --- ## [Filebeat Sonicwall module - dissect\_parsing\_error](https://discuss.elastic.co/t/filebeat-sonicwall-module-dissect-parsing-error/252708)
**Author:** [@PhilA](https://discuss.elastic.co/u/PhilA)\ **Replies:** 3\ **Last updated:** [October 21, 2020, 3:16pm UTC](https://discuss.elastic.co/t/filebeat-sonicwall-module-dissect-parsing-error/252708 "2020-10-21T15:16:49Z")
I have attempted to enable the SonicWall Filebeat module but it doesn't seem to support our logs fully. I am running v7.9.2 and looked at enabling this through ingest manager in the Kibana GUI but that doesn't seem to b… --- ## [FileBeat -\> AWS MSK over TLS Client Authentication](https://discuss.elastic.co/t/filebeat-aws-msk-over-tls-client-authentication/252709)
**Author:** [@Ravi342883](https://discuss.elastic.co/u/Ravi342883)\ **Replies:** 2\ **Last updated:** [October 21, 2020, 3:02pm UTC](https://discuss.elastic.co/t/filebeat-aws-msk-over-tls-client-authentication/252709 "2020-10-21T15:02:36Z")
Hi, Could anyone guide me with the process (or configuration) to enable TLS Client Auth while streaming data from FileBeat --\> AWS MSK What are all the params that we need to pass in FileBeat output configuration. --- ## [Unmatched responses](https://discuss.elastic.co/t/unmatched-responses/252860)
**Author:** [@toms130](https://discuss.elastic.co/u/toms130)\ **Replies:** 0\ **Last updated:** [October 21, 2020, 3:00pm UTC](https://discuss.elastic.co/t/unmatched-responses/252860 "2020-10-21T15:00:08Z")
Hi there, I've got unmatched response (and requests) in my packetbeat config, capturing http resuests Metrics logs show a lot of unmatched events 2020-10-21T16:51:39.052+0200 INFO \[monitoring\] log/log.go:145 Non-zero … --- ## [Filebeat unable to parse JSON log(contains array) output to logstash](https://discuss.elastic.co/t/filebeat-unable-to-parse-json-log-contains-array-output-to-logstash/252107)
**Author:** [@polaaditya](https://discuss.elastic.co/u/polaaditya)\ **Replies:** 2\ **Last updated:** [October 21, 2020, 2:45pm UTC](https://discuss.elastic.co/t/filebeat-unable-to-parse-json-log-contains-array-output-to-logstash/252107 "2020-10-21T14:45:28Z")
My current usage Filebeat =\> Logstash =\> Elasticsearch. Below is the error log Oct 14 14:07:21 vault-audit filebeat\[11314\]: 2020-10-13T14:07:21.055Z INFO \[publisher\] pipeline/retry.go:217 … --- ## [WinLogBeat - Windows Certificate Store](https://discuss.elastic.co/t/winlogbeat-windows-certificate-store/252727)
**Author:** [@Frank\_Barton](https://discuss.elastic.co/u/Frank_Barton)\ **Replies:** 4\ **Last updated:** [October 21, 2020, 2:40pm UTC](https://discuss.elastic.co/t/winlogbeat-windows-certificate-store/252727 "2020-10-21T14:40:28Z")
I'm experimenting with WinLogBeat, and would like to use certificate authentication on the connection to LogStash. All of my clients have certificates in the windows certificate store. Is there a way to configure winlog… --- ## [Winlogbeat - drop\_event (multiple event ID's with specific rules)](https://discuss.elastic.co/t/winlogbeat-drop-event-multiple-event-ids-with-specific-rules/252822)
**Author:** [@0xf](https://discuss.elastic.co/u/0xf)\ **Replies:** 5\ **Last updated:** [October 21, 2020, 2:06pm UTC](https://discuss.elastic.co/t/winlogbeat-drop-event-multiple-event-ids-with-specific-rules/252822 "2020-10-21T14:06:06Z")
Okay so im having a hard time solving this puzzle. Tried almost everything and i cant really solve it by myself, any ideas? So i have 2 event ID's: winlog.event\_id: 4624 winlog.event\_id: 4672 What i want to do is i w… --- ## [Kafka output support in Filebeat. Anyone using Filebeat with Kafka version \>2.2.0](https://discuss.elastic.co/t/kafka-output-support-in-filebeat-anyone-using-filebeat-with-kafka-version-2-2-0/252038)
**Author:** [@Hreidar\_Joelsson](https://discuss.elastic.co/u/Hreidar_Joelsson)\ **Replies:** 3\ **Last updated:** [October 21, 2020, 1:48pm UTC](https://discuss.elastic.co/t/kafka-output-support-in-filebeat-anyone-using-filebeat-with-kafka-version-2-2-0/252038 "2020-10-21T13:48:49Z")
Hi, I'm looking into using Filebeat as an ingress into a pipeline which is based on Kafka. I'm currently using an on-prem Kafka cluster which has brokers running Kafka version 2.5.0 but I see that the highest version sup… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=202) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=204)