# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=142 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 143 --- ## [Config check, filtering out users](https://discuss.elastic.co/t/config-check-filtering-out-users/279915)
**Author:** [@sancla](https://discuss.elastic.co/u/sancla)\ **Replies:** 1\ **Last updated:** [July 29, 2021, 12:12am UTC](https://discuss.elastic.co/t/config-check-filtering-out-users/279915 "2021-07-29T00:12:05Z")
Hey guys, I'm a bit new to elastic so I can hope a 'specialist' can help me out here. I am trying to filter out some security log records from our Exchange servers and it does not seem to be working. The config is test… --- ## [Excluded\_files doesn't seem to work](https://discuss.elastic.co/t/excluded-files-doesnt-seem-to-work/279918)
**Author:** [@prandelicious](https://discuss.elastic.co/u/prandelicious)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 10:32pm UTC](https://discuss.elastic.co/t/excluded-files-doesnt-seem-to-work/279918 "2021-07-28T22:32:07Z")
I used the default excluded\_files example: excluded\_files: - '(?i)\\.sw\[nopx\]$' - ~$ - /\\.git($|/) but it still logs temporary files created by vim like .\.swp or ~\. I also trie… --- ## [Working with gzipped files](https://discuss.elastic.co/t/working-with-gzipped-files/279644)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 1\ **Last updated:** [July 28, 2021, 7:25pm UTC](https://discuss.elastic.co/t/working-with-gzipped-files/279644 "2021-07-28T19:25:08Z")
Hi, I have to work with a lot of gzipped text files but it seems Filebeat can't read them. Is there any way to read the gzipped files in a given folder and ship those to a Kafka cluster? I saw that maybe I could work w… --- ## [Feature parity between Integrations and Filebeat modules](https://discuss.elastic.co/t/feature-parity-between-integrations-and-filebeat-modules/279714)
**Author:** [@wsnet](https://discuss.elastic.co/u/wsnet)\ **Replies:** 1\ **Last updated:** [July 28, 2021, 4:13pm UTC](https://discuss.elastic.co/t/feature-parity-between-integrations-and-filebeat-modules/279714 "2021-07-28T16:13:55Z")
Hello, In the elastic/integrations repo there are lots of issues/plans and some PRs to add more data sources into Elastic, which is great! Does Elastic intend to also add these data sources as Filebeat modules, or do w… --- ## [How to identify if filebeat has read all the logs from a specific folder?](https://discuss.elastic.co/t/how-to-identify-if-filebeat-has-read-all-the-logs-from-a-specific-folder/279857)
**Author:** [@Animesh\_Agarwal](https://discuss.elastic.co/u/Animesh_Agarwal)\ **Replies:** 1\ **Last updated:** [July 28, 2021, 4:07pm UTC](https://discuss.elastic.co/t/how-to-identify-if-filebeat-has-read-all-the-logs-from-a-specific-folder/279857 "2021-07-28T16:07:19Z")
Hi Team, Is there a way to identify if filebeat has parsed all the logs from a specific folder ? Requirement is to delete the folder automatically once filebeat has synced all the logs from that folder --- ## [Filebeat latency while reading the data](https://discuss.elastic.co/t/filebeat-latency-while-reading-the-data/279869)
**Author:** [@anjilinga](https://discuss.elastic.co/u/anjilinga)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 2:05pm UTC](https://discuss.elastic.co/t/filebeat-latency-while-reading-the-data/279869 "2021-07-28T14:05:15Z")
Hi Team, in filebeat i have given the input section with filepath and outputsection with kafka. while reading the data from file filebeat is reading in batchwise. I have not given any flush timeout, as a default it sho… --- ## [Getting custom filebeat module into production](https://discuss.elastic.co/t/getting-custom-filebeat-module-into-production/275740)
**Author:** [@kelvins](https://discuss.elastic.co/u/kelvins)\ **Replies:** 4\ **Last updated:** [July 28, 2021, 10:13am UTC](https://discuss.elastic.co/t/getting-custom-filebeat-module-into-production/275740 "2021-07-28T10:13:11Z")
I have created a new beats module (jde) which contain a couple of filesets. I have gone through the testing phases of the filebeat modules and all tests pass - Yeah! I am now trying to move this into production. To do… --- ## [Filebeat cannot connect to kibana: error loading index pattern: parsing kibana response](https://discuss.elastic.co/t/filebeat-cannot-connect-to-kibana-error-loading-index-pattern-parsing-kibana-response/279328)
**Author:** [@outranker](https://discuss.elastic.co/u/outranker)\ **Replies:** 4\ **Last updated:** [July 28, 2021, 8:12am UTC](https://discuss.elastic.co/t/filebeat-cannot-connect-to-kibana-error-loading-index-pattern-parsing-kibana-response/279328 "2021-07-28T08:12:10Z")
I have set up Kibana and Elasticsearch and they work fine so far. I have also set up Filebeat on another aws ec2 instance to send logs to Elasticsearch and monitor from Kibana But for some reason Filebeat cannot connec… --- ## [Monitoring Logstash using Metricbeat](https://discuss.elastic.co/t/monitoring-logstash-using-metricbeat/277934)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 2\ **Last updated:** [July 28, 2021, 7:04am UTC](https://discuss.elastic.co/t/monitoring-logstash-using-metricbeat/277934 "2021-07-28T07:04:25Z")
Hi I have a cluster with 4 nodes of Elasticsearch. 3 of them have Logstash and Metricbeat running for data ship and monitoring the pipelines and Logstash nodes. When I config the Metricbeat , I see Pipelines of 3 Logst… --- ## [Configuring Filebeat to use proxy for any input request that goes out](https://discuss.elastic.co/t/configuring-filebeat-to-use-proxy-for-any-input-request-that-goes-out/279485)
**Author:** [@code4purpose](https://discuss.elastic.co/u/code4purpose)\ **Replies:** 7\ **Last updated:** [July 27, 2021, 8:16pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-use-proxy-for-any-input-request-that-goes-out/279485 "2021-07-27T20:16:55Z")
I am running Elasticsearch, Kibana and Filebeats on my office windows laptop. I am trying to use filebeat -microsoft module. All outgoing http/s requests go via a proxy. How do I Configure Filebeat to use proxy for an… --- ## [The traffic ports of the mirroring session are the same](https://discuss.elastic.co/t/the-traffic-ports-of-the-mirroring-session-are-the-same/279741)
**Author:** [@chinaxushi](https://discuss.elastic.co/u/chinaxushi)\ **Replies:** 0\ **Last updated:** [July 27, 2021, 2:34pm UTC](https://discuss.elastic.co/t/the-traffic-ports-of-the-mirroring-session-are-the-same/279741 "2021-07-27T14:34:23Z")
The mirrored traffic can be transmitted through the same protocol port. How do you solve this situation ? Thank you for your attention --- ## [How to support AWS cloudwatch metric stream to Elasticsearch?](https://discuss.elastic.co/t/how-to-support-aws-cloudwatch-metric-stream-to-elasticsearch/279672)
**Author:** [@zhaoyi0113](https://discuss.elastic.co/u/zhaoyi0113)\ **Replies:** 4\ **Last updated:** [July 27, 2021, 2:04pm UTC](https://discuss.elastic.co/t/how-to-support-aws-cloudwatch-metric-stream-to-elasticsearch/279672 "2021-07-27T14:04:00Z")
I am deploying applications to AWS including various services. I am planning to use Kibana and Elasticsearch to view metrics from Cloudwatch. I know I can configure metricbeat by using AWS module which works as a crawler… --- ## [I can't get system.cpu.total.pct metrics from my metricbeats](https://discuss.elastic.co/t/i-cant-get-system-cpu-total-pct-metrics-from-my-metricbeats/279653)
**Author:** [@shell](https://discuss.elastic.co/u/shell)\ **Replies:** 9\ **Last updated:** [July 27, 2021, 2:01pm UTC](https://discuss.elastic.co/t/i-cant-get-system-cpu-total-pct-metrics-from-my-metricbeats/279653 "2021-07-27T14:01:25Z")
I'm getting into ELK stack and now I can't figure out how to get and visualize in Kibana metrics of CPU load (in any form, but I suppose the best choice is system.cpu.total.pct ) from my metricbeats. I can do this for me… --- ## [Exiting: error connecting to Kibana: fail to get the Kibana version: fail to unmarshal the response from GET](https://discuss.elastic.co/t/exiting-error-connecting-to-kibana-fail-to-get-the-kibana-version-fail-to-unmarshal-the-response-from-get/279704)
**Author:** [@Daniel.X](https://discuss.elastic.co/u/Daniel.X)\ **Replies:** 2\ **Last updated:** [July 27, 2021, 9:30am UTC](https://discuss.elastic.co/t/exiting-error-connecting-to-kibana-fail-to-get-the-kibana-version-fail-to-unmarshal-the-response-from-get/279704 "2021-07-27T09:30:31Z")
Hi there, I keep getting this error below: Kibana status api returns: json: cannot unmarshal string into Go struct field kibanaVersionResponse.version of type struct { Number string "json:"number""; Snapshot bool "json… --- ## [Filebeat Multiline Patterns not working with Cloudfoundry](https://discuss.elastic.co/t/filebeat-multiline-patterns-not-working-with-cloudfoundry/279705)
**Author:** [@checkmateasus](https://discuss.elastic.co/u/checkmateasus)\ **Replies:** 0\ **Last updated:** [July 27, 2021, 8:47am UTC](https://discuss.elastic.co/t/filebeat-multiline-patterns-not-working-with-cloudfoundry/279705 "2021-07-27T08:47:16Z")
Running Elastic Stack 7.10.2 ( Filebeat, Elastic search). The input for logs in filebeat is cloudfoundry (6.5). The filebeat runs as a container in cloud foundry. We need to read multiline messages coming from cloudfou… --- ## [Kibana logs vs filebeat kibana module](https://discuss.elastic.co/t/kibana-logs-vs-filebeat-kibana-module/279546)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 3\ **Last updated:** [July 27, 2021, 9:02am UTC](https://discuss.elastic.co/t/kibana-logs-vs-filebeat-kibana-module/279546 "2021-07-27T09:02:07Z")
I'm investigating the filebeat kibana module to ingest Kibana logs, but the kibana doc (7.13.4) seems lacking in how to configure the json format logs. The module seems to want to look in /var/log/kibana/\*\_audit.json an… --- ## [Adding query parameters to filebeat?](https://discuss.elastic.co/t/adding-query-parameters-to-filebeat/279517)
**Author:** [@gentle\_ghost](https://discuss.elastic.co/u/gentle_ghost)\ **Replies:** 3\ **Last updated:** [July 27, 2021, 9:01am UTC](https://discuss.elastic.co/t/adding-query-parameters-to-filebeat/279517 "2021-07-27T09:01:10Z")
Hello, I'm adding custom routing to improve performance and the documentation for custom routing:\_routing field | Elasticsearch Guide \[7.13\] | Elastic The documentation requires the routing value whenever indexing or u… --- ## [Filebeat apache module add\_labels process and if conditions](https://discuss.elastic.co/t/filebeat-apache-module-add-labels-process-and-if-conditions/279675)
**Author:** [@nico\_Ga](https://discuss.elastic.co/u/nico_Ga)\ **Replies:** 1\ **Last updated:** [July 27, 2021, 7:48am UTC](https://discuss.elastic.co/t/filebeat-apache-module-add-labels-process-and-if-conditions/279675 "2021-07-27T07:48:19Z")
Hi, I would like to add labels in filebeat apache module conf file with 3 conditions. I use input processors and if/then/else statements in apache.yml. Like that it's works : input: processors: - if: re… --- ## [AuditBeat: Kibana x509: certificate signed by unknown authority](https://discuss.elastic.co/t/auditbeat-kibana-x509-certificate-signed-by-unknown-authority/279622)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 2\ **Last updated:** [July 27, 2021, 7:05am UTC](https://discuss.elastic.co/t/auditbeat-kibana-x509-certificate-signed-by-unknown-authority/279622 "2021-07-27T07:05:10Z")
At first I was getting an elasticsearch error when i tried to run auditbeat setup which was like that: Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: \[error connecting to Elasticsearch at… --- ## [Palo Alto (a.k.a. panos or panw) Filebeat Compatibility with Palo Alto Version 10.1?](https://discuss.elastic.co/t/palo-alto-a-k-a-panos-or-panw-filebeat-compatibility-with-palo-alto-version-10-1/279661)
**Author:** [@jamiejackson](https://discuss.elastic.co/u/jamiejackson)\ **Replies:** 2\ **Last updated:** [July 27, 2021, 1:28am UTC](https://discuss.elastic.co/t/palo-alto-a-k-a-panos-or-panw-filebeat-compatibility-with-palo-alto-version-10-1/279661 "2021-07-27T01:28:21Z")
Hi, I was tasked with ingesting logs from Palo Alto 10.1. In my testbed, everything worked great when using the sample logs (beats/x-pack/filebeat/module/panw/panos/test at 877d8bcd176b2f5d4efd2a81846a481b94798b49 · elas… --- ## [Securing connection between Beats and Elastic Cloud](https://discuss.elastic.co/t/securing-connection-between-beats-and-elastic-cloud/279274)
**Author:** [@mi\_uber\_alles](https://discuss.elastic.co/u/mi_uber_alles)\ **Replies:** 4\ **Last updated:** [July 27, 2021, 1:25am UTC](https://discuss.elastic.co/t/securing-connection-between-beats-and-elastic-cloud/279274 "2021-07-27T01:25:48Z")
Hi All, Recently started using Elastic Cloud and it works fine. I have a Filebeat instance in a private network and it sends data to my deployment through the Internet. My question is, is there any way to make this data… --- ## [Change index name in winlogbeat](https://discuss.elastic.co/t/change-index-name-in-winlogbeat/279625)
**Author:** [@witkacy](https://discuss.elastic.co/u/witkacy)\ **Replies:** 3\ **Last updated:** [July 27, 2021, 1:03am UTC](https://discuss.elastic.co/t/change-index-name-in-winlogbeat/279625 "2021-07-27T01:03:06Z")
Hello, How can I change default name of Elasticsearch index in winlogbeat configuration? I tried this but with no success: winlogbeat.event\_logs: - name: MY-Log ignore\_older: 1h processors: - decode\_xm… --- ## [Mssql all requests](https://discuss.elastic.co/t/mssql-all-requests/279648)
**Author:** [@Sergey\_Zaguba](https://discuss.elastic.co/u/Sergey_Zaguba)\ **Replies:** 2\ **Last updated:** [July 26, 2021, 11:41pm UTC](https://discuss.elastic.co/t/mssql-all-requests/279648 "2021-07-26T23:41:38Z")
I have a Windows server with MsSQL + filebeat. I want to see in ELK all requests that are sent to all databases. Please tell me how best to do this. I have activated the mssql module in filebeat, but I still only get the… --- ## [Error](https://discuss.elastic.co/t/error/279225)
**Author:** [@TusharGarg](https://discuss.elastic.co/u/TusharGarg)\ **Replies:** 7\ **Last updated:** [July 26, 2021, 11:12pm UTC](https://discuss.elastic.co/t/error/279225 "2021-07-26T23:12:09Z")
C:\\Program Files\\Filebeat\> .\\filebeat.exe setup Start-Service filebeat Index setup finished. Loading dashboards (Kibana must be running and reachable) Exiting: error connecting to Kibana: fail to get the Kibana ver… --- ## [Filebeat converting special character into unicode value when loading into kafka](https://discuss.elastic.co/t/filebeat-converting-special-character-into-unicode-value-when-loading-into-kafka/279548)
**Author:** [@venkataraman](https://discuss.elastic.co/u/venkataraman)\ **Replies:** 3\ **Last updated:** [July 26, 2021, 5:26pm UTC](https://discuss.elastic.co/t/filebeat-converting-special-character-into-unicode-value-when-loading-into-kafka/279548 "2021-07-26T17:26:40Z")
Hello Everyone, I am trying to load data into kafka topic through filebeat. The major issue I am facing is, filebeat is converting special character (&) into Unicode value (\\u0026) when loading into kafka topic output. … --- ## [How to read custom log files using filebeat](https://discuss.elastic.co/t/how-to-read-custom-log-files-using-filebeat/276386)
**Author:** [@robertashok](https://discuss.elastic.co/u/robertashok)\ **Replies:** 12\ **Last updated:** [July 26, 2021, 3:00pm UTC](https://discuss.elastic.co/t/how-to-read-custom-log-files-using-filebeat/276386 "2021-07-26T15:00:18Z")
I have following TCS.log in seperate folder 2021/06/13 17:58:42 : INFO | Stock = TCS.NS, Date = 2002-08-12 2021/06/13 17:58:42 : INFO | Volume=212976 2021/06/13 17:58:42 : INFO | Low=38.724998474121094 2021… --- ## [Filebeat: output to kafka checks message size before compression](https://discuss.elastic.co/t/filebeat-output-to-kafka-checks-message-size-before-compression/279629)
**Author:** [@fld](https://discuss.elastic.co/u/fld)\ **Replies:** 0\ **Last updated:** [July 26, 2021, 1:00pm UTC](https://discuss.elastic.co/t/filebeat-output-to-kafka-checks-message-size-before-compression/279629 "2021-07-26T13:00:59Z")
According to filebeat document of kafka output for key max\_message\_bytesedit: The maximum permitted size of JSON-encoded messages. Bigger messages will be dropped. The default value is 1000000 (bytes). This value shoul… --- ## [MISP data in elasticsearch (filebeat)](https://discuss.elastic.co/t/misp-data-in-elasticsearch-filebeat/279489)
**Author:** [@zuzusa](https://discuss.elastic.co/u/zuzusa)\ **Replies:** 1\ **Last updated:** [July 26, 2021, 11:23am UTC](https://discuss.elastic.co/t/misp-data-in-elasticsearch-filebeat/279489 "2021-07-26T11:23:25Z")
Hi together, I am currently using a MISP in combination with elasticsearch. My goal is to have all events from the MISP also within elasticsearch. Currently the import of the MISP events to the elasticsearch is done vi… --- ## [Env variable on metricbeat.yml](https://discuss.elastic.co/t/env-variable-on-metricbeat-yml/279394)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 1\ **Last updated:** [July 26, 2021, 11:08am UTC](https://discuss.elastic.co/t/env-variable-on-metricbeat-yml/279394 "2021-07-26T11:08:53Z")
In metricbeat I am trying to use env variable to seperate out log file this is how I understood from few post here that I read. created /etc/metricbeat/env file and added HOST=myelkhost1 then edited /lib/systemd/syst… --- ## [Auditd exclude filters are adding a significant delay](https://discuss.elastic.co/t/auditd-exclude-filters-are-adding-a-significant-delay/279608)
**Author:** [@vincent509](https://discuss.elastic.co/u/vincent509)\ **Replies:** 0\ **Last updated:** [July 26, 2021, 9:55am UTC](https://discuss.elastic.co/t/auditd-exclude-filters-are-adding-a-significant-delay/279608 "2021-07-26T09:55:11Z")
I am running AuditBeat with the auditd module, and I have noticed that some filter rules are adding a significant delay between the event being created, and AuditBeat publishing the event. For example: Consider the foll… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=141) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=143)