# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=254 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 255 --- ## [Generate Kafka Topic dynamically with Modules \[Filebeat 7.5\]](https://discuss.elastic.co/t/generate-kafka-topic-dynamically-with-modules-filebeat-7-5/230126)
**Author:** [@RdrgPorto](https://discuss.elastic.co/u/RdrgPorto)\ **Replies:** 1\ **Last updated:** [April 29, 2020, 5:09am UTC](https://discuss.elastic.co/t/generate-kafka-topic-dynamically-with-modules-filebeat-7-5/230126 "2020-04-29T05:09:05Z")
Hi, everyone I use Filebeat in order to send all my data to Apache Kafka. Moreover, I use parameter fields in order to generate Kafka Topic dynamically. Here you are an example: filebeat.inputs: - type: log paths: … --- ## [How to configure FileBeat path based on LogicMonitor log](https://discuss.elastic.co/t/how-to-configure-filebeat-path-based-on-logicmonitor-log/230286)
**Author:** [@rohana23](https://discuss.elastic.co/u/rohana23)\ **Replies:** 0\ **Last updated:** [April 29, 2020, 2:47am UTC](https://discuss.elastic.co/t/how-to-configure-filebeat-path-based-on-logicmonitor-log/230286 "2020-04-29T02:47:50Z")
Hi all, I'm new to this elastic stuff. Need help on; Scenario; we need to visualize device usage/condition in Kibana FROM LogicMonitor. Where to find LogicMonitor logs directory? (yeah, knows nothing about LogicMonit… --- ## [SQS DeleteMessageRequest failed with S3 input in Filebeat](https://discuss.elastic.co/t/sqs-deletemessagerequest-failed-with-s3-input-in-filebeat/230074)
**Author:** [@nhnicwaller](https://discuss.elastic.co/u/nhnicwaller)\ **Replies:** 1\ **Last updated:** [April 29, 2020, 12:25am UTC](https://discuss.elastic.co/t/sqs-deletemessagerequest-failed-with-s3-input-in-filebeat/230074 "2020-04-29T00:25:35Z")
I'm trying to configure Filebeat for the first time with an S3 input but so far I've been unsuccessful in getting Filebeat set up correctly. It seems to read messages from SQS and S3, but when it comes time to delete the… --- ## [Winlogbeat multiple add\_fields processors throws an error](https://discuss.elastic.co/t/winlogbeat-multiple-add-fields-processors-throws-an-error/230248)
**Author:** [@nmoham](https://discuss.elastic.co/u/nmoham)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 11:57pm UTC](https://discuss.elastic.co/t/winlogbeat-multiple-add-fields-processors-throws-an-error/230248 "2020-04-28T23:57:41Z")
Some of our services write their logs to native windows Event Application logs with different SourceName(event.providers) and also to custom Event Logger, which we want add some fields to route data to appropriate indice… --- ## [AWS Uptime Geo information](https://discuss.elastic.co/t/aws-uptime-geo-information/230242)
**Author:** [@sheldonh](https://discuss.elastic.co/u/sheldonh)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 9:25pm UTC](https://discuss.elastic.co/t/aws-uptime-geo-information/230242 "2020-04-28T21:25:53Z")
I did a manual script to try and set some geo location for the geo reporting based on AWS information. I couldn't find out if way to capture this. This seems important to powering the map view. I did region, but had to m… --- ## [Heartbeat tag information](https://discuss.elastic.co/t/heartbeat-tag-information/229702)
**Author:** [@sheldonh](https://discuss.elastic.co/u/sheldonh)\ **Replies:** 4\ **Last updated:** [April 28, 2020, 9:19pm UTC](https://discuss.elastic.co/t/heartbeat-tag-information/229702 "2020-04-28T21:19:52Z")
I don't see any tags for filtering or grouping in heartbeats after configuring 100+ agents. I coded a snippet in the install script that pulls the ec2 tags and appends as yaml values. This is the format I used. Most of t… --- ## [Bug confirmation needed: Netflow module v7.6 doesn't support 4-7 byte pad in IPFIX template sets](https://discuss.elastic.co/t/bug-confirmation-needed-netflow-module-v7-6-doesnt-support-4-7-byte-pad-in-ipfix-template-sets/230245)
**Author:** [@bortok](https://discuss.elastic.co/u/bortok)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 7:36pm UTC](https://discuss.elastic.co/t/bug-confirmation-needed-netflow-module-v7-6-doesnt-support-4-7-byte-pad-in-ipfix-template-sets/230245 "2020-04-28T19:36:21Z")
Can somebody confirm this is a bug before I submit it on github? The Filebeat Netflow module ver 7.6 doesn't support 4-7 byte pad in template sets when parsing IPFIX. I have Ixia Vision E10S packet broker that can expor… --- ## [Support IAM Instance Role Permissions](https://discuss.elastic.co/t/support-iam-instance-role-permissions/229701)
**Author:** [@sheldonh](https://discuss.elastic.co/u/sheldonh)\ **Replies:** 2\ **Last updated:** [April 28, 2020, 7:15pm UTC](https://discuss.elastic.co/t/support-iam-instance-role-permissions/229701 "2020-04-28T19:15:16Z")
The configuration for modules related to AWS provide the need for access key and secret key. I want to define the permissions in the monitoring instance role and not explicitly manage a new set of access keys for grabbi… --- ## [Filebeat to Elasticsearch grok example](https://discuss.elastic.co/t/filebeat-to-elasticsearch-grok-example/230193)
**Author:** [@rkk](https://discuss.elastic.co/u/rkk)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 3:35pm UTC](https://discuss.elastic.co/t/filebeat-to-elasticsearch-grok-example/230193 "2020-04-28T15:35:28Z")
Hi, I would like to send log files using filebeat directly to elasticsearch but lines in log file are in custom string format and I would like preprocess those lines using grok or something? I cannot seem to find any e… --- ## [Auditbeat Connection Issue ElasticSearch](https://discuss.elastic.co/t/auditbeat-connection-issue-elasticsearch/230186)
**Author:** [@dcho99](https://discuss.elastic.co/u/dcho99)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 3:20pm UTC](https://discuss.elastic.co/t/auditbeat-connection-issue-elasticsearch/230186 "2020-04-28T15:20:57Z")
I'm fairly new to Elastic. When configuring my auditbeats.yml file and testing it I get this error: Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: \[Error connection to Elasticsearch http… --- ## [Cannot show ping because icmp.rtt.us no value](https://discuss.elastic.co/t/cannot-show-ping-because-icmp-rtt-us-no-value/226011)
**Author:** [@gilang\_bilbisri](https://discuss.elastic.co/u/gilang_bilbisri)\ **Replies:** 8\ **Last updated:** [April 28, 2020, 11:45am UTC](https://discuss.elastic.co/t/cannot-show-ping-because-icmp-rtt-us-no-value/226011 "2020-04-28T11:45:57Z")
Hello everyone I have a project to make some monitoring with ELK stack then for the monitoring i use grafana for user client. But i have a problems when im try to show up Ping visualization. There is no value for icmp.… --- ## [Externalization of SSL certificate for Functionbeat](https://discuss.elastic.co/t/externalization-of-ssl-certificate-for-functionbeat/230151)
**Author:** [@Ujval](https://discuss.elastic.co/u/Ujval)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 11:41am UTC](https://discuss.elastic.co/t/externalization-of-ssl-certificate-for-functionbeat/230151 "2020-04-28T11:41:29Z")
How to externalize ssl certificate configuration for Functionbeat aws lambda function to make binary agnostic of environment? --- ## [How to load external configuration file?](https://discuss.elastic.co/t/how-to-load-external-configuration-file/228757)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 4\ **Last updated:** [April 28, 2020, 10:05am UTC](https://discuss.elastic.co/t/how-to-load-external-configuration-file/228757 "2020-04-28T10:05:27Z")
hi we are trying to modularize all of the configurations as a standard. As per the https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-reloading.html link . I couldn't find a precise way of doi… --- ## [Custom index name for auditbeat](https://discuss.elastic.co/t/custom-index-name-for-auditbeat/229968)
**Author:** [@michielM](https://discuss.elastic.co/u/michielM)\ **Replies:** 5\ **Last updated:** [April 28, 2020, 9:18am UTC](https://discuss.elastic.co/t/custom-index-name-for-auditbeat/229968 "2020-04-28T09:18:04Z")
Hi everyone, I'm setting up a custom index name for auditbeat. This part has worked so far, but if I want to load the default kibana dashboards from auditbeat, I keep getting the error: "Could not locate that index-pat… --- ## [Metricbeat and advanced Prometheus queries](https://discuss.elastic.co/t/metricbeat-and-advanced-prometheus-queries/205412)
**Author:** [@hilt86](https://discuss.elastic.co/u/hilt86)\ **Replies:** 7\ **Last updated:** [April 28, 2020, 9:12am UTC](https://discuss.elastic.co/t/metricbeat-and-advanced-prometheus-queries/205412 "2020-04-28T09:12:37Z")
I'm trying to use the metricbeat prometheus module to run a query like : query: 'match\[\]' : 'sum(increase(unix\_bytes\[5m\])) by (process)' however I get an error "Unable to decode response from prometheus endpoint". Am… --- ## [How can i have multiple inputs/outputs in a single filebeat.yml](https://discuss.elastic.co/t/how-can-i-have-multiple-inputs-outputs-in-a-single-filebeat-yml/230107)
**Author:** [@opensourcengineer](https://discuss.elastic.co/u/opensourcengineer)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 8:47am UTC](https://discuss.elastic.co/t/how-can-i-have-multiple-inputs-outputs-in-a-single-filebeat-yml/230107 "2020-04-28T08:47:59Z")
i am using ES 7.5.2 version and I have installed the filebeat on syslog server. now in my syslog server we have different directory for different types of logs. lets say directory name vmware for esxi logs and xen for x… --- ## [Filebeat: multiline: introduce merge by using max-lines as condition in stead of pattern](https://discuss.elastic.co/t/filebeat-multiline-introduce-merge-by-using-max-lines-as-condition-in-stead-of-pattern/229434)
**Author:** [@williamd67](https://discuss.elastic.co/u/williamd67)\ **Replies:** 4\ **Last updated:** [April 28, 2020, 8:07am UTC](https://discuss.elastic.co/t/filebeat-multiline-introduce-merge-by-using-max-lines-as-condition-in-stead-of-pattern/229434 "2020-04-28T08:07:21Z")
Once in a while people like to merge messages into a single line not based on a pattern but based on the number of lines that have to be merged. This may be caused by not having a clear usable pattern or by just wanting … --- ## [Filebeat authentication fail](https://discuss.elastic.co/t/filebeat-authentication-fail/230024)
**Author:** [@Georgios\_Nikoloudis](https://discuss.elastic.co/u/Georgios_Nikoloudis)\ **Replies:** 9\ **Last updated:** [April 28, 2020, 7:02am UTC](https://discuss.elastic.co/t/filebeat-authentication-fail/230024 "2020-04-28T07:02:01Z")
hi after installing the filebeat to a remote server it looks that cannot authenticate to ES. ES Log ^\[\[A^\[\[A\[2020-04-27T22:00:41,318\]\[WARN \]\[o.e.x.s.a.AuthenticationService\] \[node-1\] Authentication to realm default\_na… --- ## [Prometheus Metrics data into ES via Metricbeats](https://discuss.elastic.co/t/prometheus-metrics-data-into-es-via-metricbeats/229392)
**Author:** [@Jalpesh1](https://discuss.elastic.co/u/Jalpesh1)\ **Replies:** 2\ **Last updated:** [April 28, 2020, 6:52am UTC](https://discuss.elastic.co/t/prometheus-metrics-data-into-es-via-metricbeats/229392 "2020-04-28T06:52:03Z")
Hi Team, We are exporting Metrics data from Prometheus to ES via Metricbeat but when we see data in Kibana it's showing 1 minute interval data but scraping data time-frame is 10 seconds in Prometheus . Is there any c… --- ## [How to ship multiple cloud watch log group to logstash and store them in elasticsearch cluster in individual indices](https://discuss.elastic.co/t/how-to-ship-multiple-cloud-watch-log-group-to-logstash-and-store-them-in-elasticsearch-cluster-in-individual-indices/229808)
**Author:** [@taybur.rahaman](https://discuss.elastic.co/u/taybur.rahaman)\ **Replies:** 2\ **Last updated:** [April 28, 2020, 3:57am UTC](https://discuss.elastic.co/t/how-to-ship-multiple-cloud-watch-log-group-to-logstash-and-store-them-in-elasticsearch-cluster-in-individual-indices/229808 "2020-04-28T03:57:25Z")
Actually i want to use one functionbeat labmda function for shiping all of my cloudwatch log group to logstash and after processing each of them will be stored in individual index in elastic search. currently i have be… --- ## [Aws Cloudwatch ELB metrics with more than one dimension do not have aws.tags field](https://discuss.elastic.co/t/aws-cloudwatch-elb-metrics-with-more-than-one-dimension-do-not-have-aws-tags-field/230066)
**Author:** [@martinkwge](https://discuss.elastic.co/u/martinkwge)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 3:02am UTC](https://discuss.elastic.co/t/aws-cloudwatch-elb-metrics-with-more-than-one-dimension-do-not-have-aws-tags-field/230066 "2020-04-28T03:02:41Z")
metricbeat.modules: - module: aws period: 300s metricsets: - cloudwatch metrics: - namespace: AWS/NetworkELB statistic: \["Maximum"\] name: \["UnHealthyHostCount"\] tags.resource\_type\_filter: … --- ## [Host.name field winlogbeat (FQDN) vs filebeat (shortname)](https://discuss.elastic.co/t/host-name-field-winlogbeat-fqdn-vs-filebeat-shortname/228398)
**Author:** [@jimmburton](https://discuss.elastic.co/u/jimmburton)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 2:25am UTC](https://discuss.elastic.co/t/host-name-field-winlogbeat-fqdn-vs-filebeat-shortname/228398 "2020-04-28T02:25:13Z")
I have built a brand new 7.6.2 cluster and I am testing SIEM with it as well as making some changes in the architecture and index layout from my old 6.x cluster I have updated a few times. I have a windows system where … --- ## [How to mask the data using Filebeats processors](https://discuss.elastic.co/t/how-to-mask-the-data-using-filebeats-processors/230058)
**Author:** [@elkdeveloper](https://discuss.elastic.co/u/elkdeveloper)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 1:19am UTC](https://discuss.elastic.co/t/how-to-mask-the-data-using-filebeats-processors/230058 "2020-04-28T01:19:44Z")
Dear All, 'I am new to the elk and planning to implement filebeats for custom logs in one of the application. Mask the data within the tags before sending to logstash using filebeat processors. please verify the below… --- ## [Copy winlog.user.name (if present) to user.name (if missing)](https://discuss.elastic.co/t/copy-winlog-user-name-if-present-to-user-name-if-missing/229313)
**Author:** [@\_finack](https://discuss.elastic.co/u/_finack)\ **Replies:** 5\ **Last updated:** [April 27, 2020, 9:39pm UTC](https://discuss.elastic.co/t/copy-winlog-user-name-if-present-to-user-name-if-missing/229313 "2020-04-27T21:39:59Z")
For normalization purposes, I would like to copy the contents of winlog.user.name to user.name if the former is present but the latter is not. I tried the following, but it does not work: - copy\_fields: when: … --- ## [Filebeat with container discovery stucks once a day](https://discuss.elastic.co/t/filebeat-with-container-discovery-stucks-once-a-day/230031)
**Author:** [@r2r2](https://discuss.elastic.co/u/r2r2)\ **Replies:** 0\ **Last updated:** [April 27, 2020, 9:03pm UTC](https://discuss.elastic.co/t/filebeat-with-container-discovery-stucks-once-a-day/230031 "2020-04-27T21:03:16Z")
Hello! I have 3 containers of docker.elastic.co/beats/filebeat:7.6.1 in Nomad. They stop to send logs once a day in random time and begin to waste cpu iowait. Restart of container helps approximatelly for a day. Config… --- ## [Which user role to be selected for User name which will be configured in Winlogbeat configuration?](https://discuss.elastic.co/t/which-user-role-to-be-selected-for-user-name-which-will-be-configured-in-winlogbeat-configuration/230007)
**Author:** [@santhiprakash](https://discuss.elastic.co/u/santhiprakash)\ **Replies:** 1\ **Last updated:** [April 27, 2020, 7:36pm UTC](https://discuss.elastic.co/t/which-user-role-to-be-selected-for-user-name-which-will-be-configured-in-winlogbeat-configuration/230007 "2020-04-27T19:36:10Z")
Hi Folks, After trying for a few days, I am able to set up the Elastic server with Kibana in the cluster. To Deploy Beats (Winlogbeat) on desktops to send logs to Elastic or Logstash, Which user role to be selected for… --- ## [Sending HTTP POST request from the metricbeat script processor](https://discuss.elastic.co/t/sending-http-post-request-from-the-metricbeat-script-processor/229985)
**Author:** [@musician](https://discuss.elastic.co/u/musician)\ **Replies:** 0\ **Last updated:** [April 27, 2020, 3:47pm UTC](https://discuss.elastic.co/t/sending-http-post-request-from-the-metricbeat-script-processor/229985 "2020-04-27T15:47:19Z")
Hello, I am using metricbeat 7.6.2 on Windows 2012. Is there any way to send an HTTP POST request from the script processor? The use case is that if the CPU usage for a specific process is over a specified threshold, an … --- ## [Issue with parsing nginx log](https://discuss.elastic.co/t/issue-with-parsing-nginx-log/229967)
**Author:** [@Samuel\_Stanislav](https://discuss.elastic.co/u/Samuel_Stanislav)\ **Replies:** 2\ **Last updated:** [April 27, 2020, 3:36pm UTC](https://discuss.elastic.co/t/issue-with-parsing-nginx-log/229967 "2020-04-27T15:36:59Z")
Hello , I m sending Nginx logs via filebeat -\> elastic search -\> Kibana . But already have issue with some logs . It s look like this type of log is parsing without any problem : 66.249.76.123 - - \[24/Apr/2020:17:24:5… --- ## [Collect logs from shibboleth sp outside stdout](https://discuss.elastic.co/t/collect-logs-from-shibboleth-sp-outside-stdout/229908)
**Author:** [@bopedibop](https://discuss.elastic.co/u/bopedibop)\ **Replies:** 2\ **Last updated:** [April 27, 2020, 3:07pm UTC](https://discuss.elastic.co/t/collect-logs-from-shibboleth-sp-outside-stdout/229908 "2020-04-27T15:07:48Z")
Hi I have a container that is not sending logs to stdout, if i try nothing will show up. So how can I config filebeat to collect logs from another container outside stdout. Do I have to put the logs on the host and ha… --- ## [Error : panic: runtime error: makeslice: len out of range](https://discuss.elastic.co/t/error-panic-runtime-error-makeslice-len-out-of-range/229840)
**Author:** [@JavaDeveloper](https://discuss.elastic.co/u/JavaDeveloper)\ **Replies:** 1\ **Last updated:** [April 27, 2020, 2:56pm UTC](https://discuss.elastic.co/t/error-panic-runtime-error-makeslice-len-out-of-range/229840 "2020-04-27T14:56:41Z")
Hello. I'm using one of the community filebeats called mqttbeat written by Go language. I configured it, so that it connects to my own mqtt broker and ships data to the elasticsearch properly. However, after 1 hour, t… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=253) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=255)