# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=183 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 184 --- ## [What regex to use to exclude 30s status messages](https://discuss.elastic.co/t/what-regex-to-use-to-exclude-30s-status-messages/262219)
**Author:** [@Peter\_Boos](https://discuss.elastic.co/u/Peter_Boos)\ **Replies:** 0\ **Last updated:** [January 26, 2021, 10:46am UTC](https://discuss.elastic.co/t/what-regex-to-use-to-exclude-30s-status-messages/262219 "2021-01-26T10:46:33Z")
What should be the working regex to exclude the 30s filebeat status updates The 30s status starts with, (but are quite a bit longer) : 2021-01-26T05:29:10.562-0500#011INFO#011\[monitoring\]#011log/log.go:145#011Non-zero… --- ## [Filtering out unwanted infobeat hartbeat 30s](https://discuss.elastic.co/t/filtering-out-unwanted-infobeat-hartbeat-30s/262202)
**Author:** [@Peter\_Boos](https://discuss.elastic.co/u/Peter_Boos)\ **Replies:** 0\ **Last updated:** [January 26, 2021, 8:11am UTC](https://discuss.elastic.co/t/filtering-out-unwanted-infobeat-hartbeat-30s/262202 "2021-01-26T08:11:09Z")
I'd like to remove the log that filebeat generates every 30s, It starts with : 021-01-18T08:29:59.656-0500#011INFO#011\[monitoring\]#011log/log.go:145#011Non-zero metrics in the last 30s#011{"monitoring": {"metrics": {"b… --- ## [Adding config for custom log integration on an agent](https://discuss.elastic.co/t/adding-config-for-custom-log-integration-on-an-agent/262181)
**Author:** [@Tim\_Estes](https://discuss.elastic.co/u/Tim_Estes)\ **Replies:** 5\ **Last updated:** [January 26, 2021, 4:55am UTC](https://discuss.elastic.co/t/adding-config-for-custom-log-integration-on-an-agent/262181 "2021-01-26T04:55:08Z")
Hello, I'm trying to index json logs with a Fleet-managed agent using a custom log configuration. I successfully downloaded the agent to my computer and specified a path where the log files live. However, when I genera… --- ## [Is there any Latency module/merticset available across beats family](https://discuss.elastic.co/t/is-there-any-latency-module-merticset-available-across-beats-family/261804)
**Author:** [@Ishaan](https://discuss.elastic.co/u/Ishaan)\ **Replies:** 10\ **Last updated:** [January 25, 2021, 10:14pm UTC](https://discuss.elastic.co/t/is-there-any-latency-module-merticset-available-across-beats-family/261804 "2021-01-25T22:14:30Z")
Hello, I would like to know if there is any Latency module/metricset available across beats family to pull latency data from the different OS platforms? Thanks --- ## [Path config for custom log integration?](https://discuss.elastic.co/t/path-config-for-custom-log-integration/262158)
**Author:** [@Tim\_Estes](https://discuss.elastic.co/u/Tim_Estes)\ **Replies:** 4\ **Last updated:** [January 25, 2021, 8:16pm UTC](https://discuss.elastic.co/t/path-config-for-custom-log-integration/262158 "2021-01-25T20:16:53Z")
Hello, I'm trying out the elastic agent feature in a test cluster but I haven't been able to figure out how to ship custom logs to it. In the integration, I've specified the log path to be path/to/my/logs: I was abl… --- ## [Ingest the same file multiple times](https://discuss.elastic.co/t/ingest-the-same-file-multiple-times/262142)
**Author:** [@Emiliano\_De\_Angelis](https://discuss.elastic.co/u/Emiliano_De_Angelis)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 4:42pm UTC](https://discuss.elastic.co/t/ingest-the-same-file-multiple-times/262142 "2021-01-25T16:42:33Z")
hello, I would need to read and forward the same file to elastic twice, duplicating the information within elastic but on which to add a different field. how can i parse the same file twice? thanks --- ## [Does dissect processor in beats support greedy/lazy match?](https://discuss.elastic.co/t/does-dissect-processor-in-beats-support-greedy-lazy-match/261616)
**Author:** [@cy-zheng](https://discuss.elastic.co/u/cy-zheng)\ **Replies:** 3\ **Last updated:** [January 25, 2021, 4:33pm UTC](https://discuss.elastic.co/t/does-dissect-processor-in-beats-support-greedy-lazy-match/261616 "2021-01-25T16:33:17Z")
I'm using dissect processor to parse a value like /var/log/containers/calico-node-ltx4t\_kube-system\_calico-node-21e03abb29adcec39f69e3f76c16eb2b3ed4dd1a51675279de5cedc59bdce56b.log using tokenizer:/var/log/containers/%… --- ## [Packetbeat "hanging" on high speed network](https://discuss.elastic.co/t/packetbeat-hanging-on-high-speed-network/262117)
**Author:** [@Antonio\_Augusto\_Sant](https://discuss.elastic.co/u/Antonio_Augusto_Sant)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 1:56pm UTC](https://discuss.elastic.co/t/packetbeat-hanging-on-high-speed-network/262117 "2021-01-25T13:56:50Z")
Gentlemen, I've configured packetbeat on a machine with plenty resources (72GB RAM, 24 cores Intel Xeon X5670), and its listening on two 10gb interfaces, that combined are reaching about 1gbps during peak hours. On this… --- ## [How to make custom template out of default Filebeat template](https://discuss.elastic.co/t/how-to-make-custom-template-out-of-default-filebeat-template/262075)
**Author:** [@ORich](https://discuss.elastic.co/u/ORich)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 10:06am UTC](https://discuss.elastic.co/t/how-to-make-custom-template-out-of-default-filebeat-template/262075 "2021-01-25T10:06:15Z")
Dear all, I setup a new template as follow to distinguish filebeat file names. The problem I have is that this new template has no mapping configured ... how can I make this new index template inherit from default fileb… --- ## [Logs mixed - same path on different servers](https://discuss.elastic.co/t/logs-mixed-same-path-on-different-servers/261799)
**Author:** [@yodog](https://discuss.elastic.co/u/yodog)\ **Replies:** 1\ **Last updated:** [January 25, 2021, 12:44pm UTC](https://discuss.elastic.co/t/logs-mixed-same-path-on-different-servers/261799 "2021-01-25T12:44:09Z")
i have 2 services running on apache: mail + drive the logs are getting mixed because they are on the same path (but on different servers). - type: log tags: \['drive'\] fields\_under\_root: true fields: '@metadat… --- ## [Import filebeat ingest pipeline](https://discuss.elastic.co/t/import-filebeat-ingest-pipeline/262104)
**Author:** [@Thomas74](https://discuss.elastic.co/u/Thomas74)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 12:36pm UTC](https://discuss.elastic.co/t/import-filebeat-ingest-pipeline/262104 "2021-01-25T12:36:02Z")
Hi, I'm upgrading my cluster from version 7.4.0 to 7.10.2. I noticed that there's grok pattern warning, I think due to old filebeat pipeline : Terraform is used to push pipeline in our cluster but no json versions … --- ## [Sophos log configuration](https://discuss.elastic.co/t/sophos-log-configuration/262099)
**Author:** [@santosh](https://discuss.elastic.co/u/santosh)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 11:52am UTC](https://discuss.elastic.co/t/sophos-log-configuration/262099 "2021-01-25T11:52:48Z")
I have been trying to pull the logs from Sophos device with the following configuration but looks like something is wrong in the configuration as I can't see any log messages in the Kibana dashboard. Any pointers? I have… --- ## [Different index name based on metricset and backend logic of metricbeat](https://discuss.elastic.co/t/different-index-name-based-on-metricset-and-backend-logic-of-metricbeat/262070)
**Author:** [@navin1093](https://discuss.elastic.co/u/navin1093)\ **Replies:** 0\ **Last updated:** [January 25, 2021, 8:43am UTC](https://discuss.elastic.co/t/different-index-name-based-on-metricset-and-backend-logic-of-metricbeat/262070 "2021-01-25T08:43:49Z")
Hi experts, My design is metricbeat -\> logstash -\> elasticsearch -\> kibana Would like your adive, should i have different index for different metric set? For isntance, performance-cpu, performance-memory,... is htis … --- ## [What line indicates that filebeat is creating an index in Elasticsearch](https://discuss.elastic.co/t/what-line-indicates-that-filebeat-is-creating-an-index-in-elasticsearch/261791)
**Author:** [@rajeshkp](https://discuss.elastic.co/u/rajeshkp)\ **Replies:** 2\ **Last updated:** [January 25, 2021, 4:34am UTC](https://discuss.elastic.co/t/what-line-indicates-that-filebeat-is-creating-an-index-in-elasticsearch/261791 "2021-01-25T04:34:09Z")
What line indicates that Filebeat 7.7.0 is creating an index in Elasticsearch. We are trying to debug the fact that its not creating an Elasticsearch index, but the harvester fails saying the index is missing. Isn't it t… --- ## [Elastic ILM does not work as expected after upgrading Elastic](https://discuss.elastic.co/t/elastic-ilm-does-not-work-as-expected-after-upgrading-elastic/261554)
**Author:** [@Ali\_Nazemian](https://discuss.elastic.co/u/Ali_Nazemian)\ **Replies:** 2\ **Last updated:** [January 25, 2021, 12:36am UTC](https://discuss.elastic.co/t/elastic-ilm-does-not-work-as-expected-after-upgrading-elastic/261554 "2021-01-25T00:36:09Z")
After I have upgraded my Elasticsearch cluster from 7.9 to 7.10.1 my ILM for filebeat does not work as expected. I can see that the corresponding indices are being marked as Warm when I check the index management, but I … --- ## [\[Solved\] Data path already locked by another beat](https://discuss.elastic.co/t/solved-data-path-already-locked-by-another-beat/262018)
**Author:** [@jfasajr](https://discuss.elastic.co/u/jfasajr)\ **Replies:** 1\ **Last updated:** [January 24, 2021, 2:33am UTC](https://discuss.elastic.co/t/solved-data-path-already-locked-by-another-beat/262018 "2021-01-24T02:33:40Z")
I'm not particularly sure what to do about the following error. I've read a few things and everyone seems to say that it's either a failed shutdown or second instance. However none of my troubleshooting has rendered resu… --- ## [\[Winlogbeat 7.10.1 dashboards\]: could not locate some index pattern fields](https://discuss.elastic.co/t/winlogbeat-7-10-1-dashboards-could-not-locate-some-index-pattern-fields/261984)
**Author:** [@ethical20](https://discuss.elastic.co/u/ethical20)\ **Replies:** 0\ **Last updated:** [January 23, 2021, 9:27am UTC](https://discuss.elastic.co/t/winlogbeat-7-10-1-dashboards-could-not-locate-some-index-pattern-fields/261984 "2021-01-23T09:27:20Z")
Hello, I am using Elasticsearch, kibana, logstash, and winlogbeat, all version 7.10.1 I configured winlogbeat and run: .\\winlogbeat.exe setup -e and I had no error in my winlogbeat logs, but in kibana there are some da… --- ## [Filebeat AWS s3 input -- SQS required?](https://discuss.elastic.co/t/filebeat-aws-s3-input-sqs-required/261963)
**Author:** [@klutch27](https://discuss.elastic.co/u/klutch27)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 10:55pm UTC](https://discuss.elastic.co/t/filebeat-aws-s3-input-sqs-required/261963 "2021-01-22T22:55:21Z")
I have log files in s3, and I'd like to use filebeat to ship them to logstash. I'm not using AWS SQS, so I have no queue\_url. Is it possible for me to consume log files from my s3 bucket, without using SQS? I was hoping… --- ## [Can't get filebeat log data into Elasticsearch](https://discuss.elastic.co/t/cant-get-filebeat-log-data-into-elasticsearch/261943)
**Author:** [@thnk2wn](https://discuss.elastic.co/u/thnk2wn)\ **Replies:** 3\ **Last updated:** [January 22, 2021, 6:50pm UTC](https://discuss.elastic.co/t/cant-get-filebeat-log-data-into-elasticsearch/261943 "2021-01-22T18:50:45Z")
I'm trying out file beats but can't seem to get the data into Elasticsearch. I don't see issues on the filebeat side but don't see the data in Elasticsearch. My demo repo is at https://github.com/thnk2wn/elastic-logging-… --- ## [Changing Statistic in metricbeat](https://discuss.elastic.co/t/changing-statistic-in-metricbeat/261918)
**Author:** [@rudraram](https://discuss.elastic.co/u/rudraram)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 1:24pm UTC](https://discuss.elastic.co/t/changing-statistic-in-metricbeat/261918 "2021-01-22T13:24:57Z")
Hello folks, I am trying to use metricbeat - lambda metricset to import Lambda metrics to ELK. Unfortunately all the metrics are 'averaged' while i need support for 'Sum' for few metrics I tried to add following to the … --- ## [Dockerized Filebeat monitor mounted host syslog](https://discuss.elastic.co/t/dockerized-filebeat-monitor-mounted-host-syslog/261923)
**Author:** [@d01](https://discuss.elastic.co/u/d01)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 2:07pm UTC](https://discuss.elastic.co/t/dockerized-filebeat-monitor-mounted-host-syslog/261923 "2021-01-22T14:07:25Z")
I have filebeat set up along elasticsearch, logstash, kibana and other containers in docker using a docker-compose file. To monitor the host with this filebeat instance as well, I have the host /var/log/syslog mounted in… --- ## [Filebeat permissions](https://discuss.elastic.co/t/filebeat-permissions/261912)
**Author:** [@mcamara](https://discuss.elastic.co/u/mcamara)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 12:38pm UTC](https://discuss.elastic.co/t/filebeat-permissions/261912 "2021-01-22T12:38:04Z")
Hello, I am coming to ask for your help, for a problem I encountered with Filebeat. I installed the agent with user root and would like other users to be able to view the logs and conf. But by modifying the conf filebea… --- ## [Connect Prometheus metrics in Kafka to Metricbeat](https://discuss.elastic.co/t/connect-prometheus-metrics-in-kafka-to-metricbeat/261880)
**Author:** [@milanvdm](https://discuss.elastic.co/u/milanvdm)\ **Replies:** 0\ **Last updated:** [January 22, 2021, 7:48am UTC](https://discuss.elastic.co/t/connect-prometheus-metrics-in-kafka-to-metricbeat/261880 "2021-01-22T07:48:57Z")
We are using a provisioned OpenShift cluster which provides its system metrics from Prometheus to a Kafka topic. Since we are not able to run privileged daemonsets, I would like to know if there is a way to still ingest… --- ## [Filebeat not exposing the name](https://discuss.elastic.co/t/filebeat-not-exposing-the-name/261854)
**Author:** [@cwiechmann](https://discuss.elastic.co/u/cwiechmann)\ **Replies:** 0\ **Last updated:** [January 21, 2021, 8:48pm UTC](https://discuss.elastic.co/t/filebeat-not-exposing-the-name/261854 "2021-01-21T20:48:23Z")
Hi All, I'm running Filebeat (7.10.2) in a Docker-Container and wanted to switch from Internal Monitoring to Metricbeat using the Beat module. It works fine so far, but the name configured in filebeat.yml is not used. … --- ## [Data loss prevention?](https://discuss.elastic.co/t/data-loss-prevention/261707)
**Author:** [@Tuckson](https://discuss.elastic.co/u/Tuckson)\ **Replies:** 7\ **Last updated:** [January 21, 2021, 3:47pm UTC](https://discuss.elastic.co/t/data-loss-prevention/261707 "2021-01-21T15:47:01Z")
Hi, Unfortunately I am have issues with my platform. This means sometimes my beats cannot send data to my logstash/elasticsearch anymore. They keep trying of course, but this may take an hour or even 2. We are searchi… --- ## [\[HELP\] cannot automate rollovers with filebeat](https://discuss.elastic.co/t/help-cannot-automate-rollovers-with-filebeat/261809)
**Author:** [@lachezar.uzunov](https://discuss.elastic.co/u/lachezar.uzunov)\ **Replies:** 0\ **Last updated:** [January 21, 2021, 3:06pm UTC](https://discuss.elastic.co/t/help-cannot-automate-rollovers-with-filebeat/261809 "2021-01-21T15:06:01Z")
Hello everyone. I am trying to create automation rollup system to archive and delete checkpoint firewall logs based on conditions. The problem I face is not being able to automate my filebeat to write to the new create… --- ## [Error Trying to package a custom made beat](https://discuss.elastic.co/t/error-trying-to-package-a-custom-made-beat/261752)
**Author:** [@Avi\_Adnahani](https://discuss.elastic.co/u/Avi_Adnahani)\ **Replies:** 0\ **Last updated:** [January 21, 2021, 9:25am UTC](https://discuss.elastic.co/t/error-trying-to-package-a-custom-made-beat/261752 "2021-01-21T09:25:45Z")
Hello, I am trying to create a new custom beat, after running "mage -v package" I get the error: sh: 1: build/mage-linux-amd64: not found Error: failed building for linux/386: exit status 127 after investigating I fou… --- ## [Performace Tuning to avoid Latency](https://discuss.elastic.co/t/performace-tuning-to-avoid-latency/261717)
**Author:** [@Ishaan](https://discuss.elastic.co/u/Ishaan)\ **Replies:** 3\ **Last updated:** [January 21, 2021, 5:27am UTC](https://discuss.elastic.co/t/performace-tuning-to-avoid-latency/261717 "2021-01-21T05:27:49Z")
How can we do performance tuning in Metricbeat Configuration file to avoid any latency? or What are the best practices one should follow to avoid any latency in production? Any recommendation or suggestion would be help… --- ## [Could I drop @timestamp and @metadata in a event before sending event to Kafka?](https://discuss.elastic.co/t/could-i-drop-timestamp-and-metadata-in-a-event-before-sending-event-to-kafka/261719)
**Author:** [@cy-zheng](https://discuss.elastic.co/u/cy-zheng)\ **Replies:** 1\ **Last updated:** [January 21, 2021, 4:40am UTC](https://discuss.elastic.co/t/could-i-drop-timestamp-and-metadata-in-a-event-before-sending-event-to-kafka/261719 "2021-01-21T04:40:23Z")
Could I drop @timestamp and @metadata in a event before sending event to Kafka? --- ## [Filebeat cisco asa module source and destination IP showing reversed](https://discuss.elastic.co/t/filebeat-cisco-asa-module-source-and-destination-ip-showing-reversed/261327)
**Author:** [@ajesh](https://discuss.elastic.co/u/ajesh)\ **Replies:** 2\ **Last updated:** [January 21, 2021, 4:25am UTC](https://discuss.elastic.co/t/filebeat-cisco-asa-module-source-and-destination-ip-showing-reversed/261327 "2021-01-21T04:25:23Z")
Hi Team, We are using the filebeat cisco asa module in filebeat to parse the data from cisco asa firewalls. One issues we noticed is the source IP and destination IP are actually showing as reversed when its parsed. Fi… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=182) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=184)