# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=289 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 290 --- ## [Filebeats parsing docker logs with max-size limit](https://discuss.elastic.co/t/filebeats-parsing-docker-logs-with-max-size-limit/213015)
**Author:** [@ml\_docker\_captain](https://discuss.elastic.co/u/ml_docker_captain)\ **Replies:** 3\ **Last updated:** [December 27, 2019, 12:56am UTC](https://discuss.elastic.co/t/filebeats-parsing-docker-logs-with-max-size-limit/213015 "2019-12-27T00:56:01Z")
Hy, I have few services that create a lot of logs and I want to start using ES + logfile to parse them. Now, since once logs are parsed and send to ES I don't need them(and since I'm forced to use on premise server with … --- ## [If I change one of the .yml files, do I need to run \[metricbeat or filebeat\] setup each time to reindex, etc?](https://discuss.elastic.co/t/if-i-change-one-of-the-yml-files-do-i-need-to-run-metricbeat-or-filebeat-setup-each-time-to-reindex-etc/213113)
**Author:** [@Shorthills](https://discuss.elastic.co/u/Shorthills)\ **Replies:** 0\ **Last updated:** [December 27, 2019, 12:10am UTC](https://discuss.elastic.co/t/if-i-change-one-of-the-yml-files-do-i-need-to-run-metricbeat-or-filebeat-setup-each-time-to-reindex-etc/213113 "2019-12-27T00:10:24Z")
Question above says it all. I ask b/c I'm trying to troubleshoot my own issues and it would be helpful to know. Thanks in advance. --- ## [Can't monitor kafka module metrics like producer, consumer, broker in 6.8.1 and later?](https://discuss.elastic.co/t/cant-monitor-kafka-module-metrics-like-producer-consumer-broker-in-6-8-1-and-later/213044)
**Author:** [@17earlgrey](https://discuss.elastic.co/u/17earlgrey)\ **Replies:** 1\ **Last updated:** [December 26, 2019, 2:59pm UTC](https://discuss.elastic.co/t/cant-monitor-kafka-module-metrics-like-producer-consumer-broker-in-6-8-1-and-later/213044 "2019-12-26T14:59:23Z")
https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-module-kafka.html If you look at the link here, it seems that earlier versions provided the functionality (consumer, producer, broker) in beta. htt… --- ## [Filebeat in Kubernetes does not inclue Kubernetes metadata on node restart](https://discuss.elastic.co/t/filebeat-in-kubernetes-does-not-inclue-kubernetes-metadata-on-node-restart/213019)
**Author:** [@Andrii\_Litvinov](https://discuss.elastic.co/u/Andrii_Litvinov)\ **Replies:** 0\ **Last updated:** [December 25, 2019, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-in-kubernetes-does-not-inclue-kubernetes-metadata-on-node-restart/213019 "2019-12-25T15:22:13Z")
I am running filebeat in k8s cluster with one etcd/control plane and one worker nodes. My filebeat configuration looks as follows: setup.ilm.enabled: false filebeat.inputs: - type: container paths: - /var/log/cont… --- ## [ILM parameters in config](https://discuss.elastic.co/t/ilm-parameters-in-config/212885)
**Author:** [@torten](https://discuss.elastic.co/u/torten)\ **Replies:** 4\ **Last updated:** [December 24, 2019, 12:03pm UTC](https://discuss.elastic.co/t/ilm-parameters-in-config/212885 "2019-12-24T12:03:06Z")
Good day, I want to set up ILM rollover\_alias and policy\_name parameters from the event fields, but no luck, all I got is errors like: \`ERROR instance/beat.go:878 Exiting: failed to read the ilm rollover … --- ## [Event size from beats](https://discuss.elastic.co/t/event-size-from-beats/212860)
**Author:** [@Robin020](https://discuss.elastic.co/u/Robin020)\ **Replies:** 2\ **Last updated:** [December 24, 2019, 9:37am UTC](https://discuss.elastic.co/t/event-size-from-beats/212860 "2019-12-24T09:37:11Z")
Hi, I am trying to found out what the size is of an event of Metricbeat or Packetbeat etc. Does somebody know where to find documentation about the size of events? I only found something of APM but nothing about the B… --- ## [Found Duplicated HTTP Events in Kubernetes Cluster](https://discuss.elastic.co/t/found-duplicated-http-events-in-kubernetes-cluster/212909)
**Author:** [@CoolDarran](https://discuss.elastic.co/u/CoolDarran)\ **Replies:** 1\ **Last updated:** [December 24, 2019, 4:04am UTC](https://discuss.elastic.co/t/found-duplicated-http-events-in-kubernetes-cluster/212909 "2019-12-24T04:04:20Z")
Hi, We're using Packetbeat to capture http traffic in Kubernetes Cluster. We've observed that there are duplicated http events even set ignore\_outgoing to true. Example as blow: same network.community\_id, source.ip,… --- ## [Filebeat stops proccessing s3 input with no error](https://discuss.elastic.co/t/filebeat-stops-proccessing-s3-input-with-no-error/212590)
**Author:** [@Ronin](https://discuss.elastic.co/u/Ronin)\ **Replies:** 3\ **Last updated:** [December 23, 2019, 9:42pm UTC](https://discuss.elastic.co/t/filebeat-stops-proccessing-s3-input-with-no-error/212590 "2019-12-23T21:42:52Z")
Using filebeat 7.4.2 and stack monitoring we see where the throughput of filebeat drops to zero and the only abnormal log message is: 2019-12-20T03:47:31.711Z INFO \[s3\] s3/input.go:293 Message visibility timeout updated… --- ## [Dynamic Web Page Ingestion](https://discuss.elastic.co/t/dynamic-web-page-ingestion/212879)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 2\ **Last updated:** [December 23, 2019, 7:54pm UTC](https://discuss.elastic.co/t/dynamic-web-page-ingestion/212879 "2019-12-23T19:54:09Z")
is there any way to download dynamic web page with search results to ingest? I need to download a dynamic web page with search results? --- ## [Test network data to test beats inputs](https://discuss.elastic.co/t/test-network-data-to-test-beats-inputs/212890)
**Author:** [@bitsutah](https://discuss.elastic.co/u/bitsutah)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 7:37pm UTC](https://discuss.elastic.co/t/test-network-data-to-test-beats-inputs/212890 "2019-12-23T19:37:58Z")
Is there a standard way to test a Beats input (like say, Graylog or Logstash) to make sure that an installation or configuration is working as expected? I would think that tcpreplay with a known-good pcap of Filebeat or… --- ## [Kubernetes/Filebeat - How to Handle JSON Logging for some containers](https://discuss.elastic.co/t/kubernetes-filebeat-how-to-handle-json-logging-for-some-containers/212888)
**Author:** [@Justin\_Seiser](https://discuss.elastic.co/u/Justin_Seiser)\ **Replies:** 0\ **Last updated:** [December 23, 2019, 7:11pm UTC](https://discuss.elastic.co/t/kubernetes-filebeat-how-to-handle-json-logging-for-some-containers/212888 "2019-12-23T19:11:12Z")
Hello, I understand the basic premise, I need to configure auto discover, and then configure different filters within that, to specify how to handle the logs. I have not been able to find a working example. We have mu… --- ## [Filter a specific text from a file](https://discuss.elastic.co/t/filter-a-specific-text-from-a-file/212737)
**Author:** [@mancharagopan](https://discuss.elastic.co/u/mancharagopan)\ **Replies:** 4\ **Last updated:** [December 23, 2019, 6:07pm UTC](https://discuss.elastic.co/t/filter-a-specific-text-from-a-file/212737 "2019-12-23T18:07:39Z")
I have a html file and i need to ingest only a specific line from that file to elasticsearch through logstash. Is there a way to do it? How do i filter only a line? --- ## [MetricBeat HTTP Module giving "invalid character '\<' looking for beginning of value"](https://discuss.elastic.co/t/metricbeat-http-module-giving-invalid-character-looking-for-beginning-of-value/212840)
**Author:** [@Rahul12](https://discuss.elastic.co/u/Rahul12)\ **Replies:** 1\ **Last updated:** [December 23, 2019, 5:25pm UTC](https://discuss.elastic.co/t/metricbeat-http-module-giving-invalid-character-looking-for-beginning-of-value/212840 "2019-12-23T17:25:17Z")
Hi, I am using Metricbeat HTTP module to check the availability of Urls. HTTP module working fine when I tested it with localhost:9200 url. But when I am using any custom URL it is giving me error. "invalid character … --- ## [How to produce beats that will ingest ecs stats into elastic](https://discuss.elastic.co/t/how-to-produce-beats-that-will-ingest-ecs-stats-into-elastic/212819)
**Author:** [@vanshika\_agrawal](https://discuss.elastic.co/u/vanshika_agrawal)\ **Replies:** 1\ **Last updated:** [December 23, 2019, 5:01pm UTC](https://discuss.elastic.co/t/how-to-produce-beats-that-will-ingest-ecs-stats-into-elastic/212819 "2019-12-23T17:01:23Z")
need help on how to produce beats that will ingest tor ecs stats into elastic so I can monitor them. --- ## [MSSSQL metricbeat module](https://discuss.elastic.co/t/msssql-metricbeat-module/212292)
**Author:** [@Aleksandar](https://discuss.elastic.co/u/Aleksandar)\ **Replies:** 2\ **Last updated:** [December 23, 2019, 9:52am UTC](https://discuss.elastic.co/t/msssql-metricbeat-module/212292 "2019-12-23T09:52:13Z")
Hi, We want to use metricbeat mssql module, but do not know minimum priviliges for sql user. Our version MB 7.4.2, Best regards, Aleksandar --- ## [New hosts don't show in the filter](https://discuss.elastic.co/t/new-hosts-dont-show-in-the-filter/211570)
**Author:** [@NogNeetMachinaal](https://discuss.elastic.co/u/NogNeetMachinaal)\ **Replies:** 1\ **Last updated:** [December 21, 2019, 4:43pm UTC](https://discuss.elastic.co/t/new-hosts-dont-show-in-the-filter/211570 "2019-12-21T16:43:38Z")
Team, I have installed auditbeat and filebeat to 2 Ubuntu hosts (on top of the 5 already done). I have used the same yml-config for all 7 hosts. These 2 new hosts don't show in the filter-by-agent.hostname when filebe… --- ## [Beats 7.5.x](https://discuss.elastic.co/t/beats-7-5-x/212388)
**Author:** [@shoopdas](https://discuss.elastic.co/u/shoopdas)\ **Replies:** 1\ **Last updated:** [December 21, 2019, 11:55am UTC](https://discuss.elastic.co/t/beats-7-5-x/212388 "2019-12-21T11:55:41Z")
Heya all, new here. Looking at the changelog on fir beats 7.5.0, more specifically the breaking change for umask for files. Is there any way of giving the 'other' users read permissions? WIth umask beign set to 0027, o… --- ## [Filebeat 6.8 changing floating point zero to integer in json](https://discuss.elastic.co/t/filebeat-6-8-changing-floating-point-zero-to-integer-in-json/211903)
**Author:** [@Zulu](https://discuss.elastic.co/u/Zulu)\ **Replies:** 4\ **Last updated:** [December 21, 2019, 2:23am UTC](https://discuss.elastic.co/t/filebeat-6-8-changing-floating-point-zero-to-integer-in-json/211903 "2019-12-21T02:23:56Z")
Hi, I am using filebeat to process telemetry data in json format from my application. The application generates files with json objects. When I need to log the floating point number zero, I log it as 0.0 with the intent… --- ## [Streaming Binary files's decoded data to elastic](https://discuss.elastic.co/t/streaming-binary-filess-decoded-data-to-elastic/212712)
**Author:** [@liron\_gofberg](https://discuss.elastic.co/u/liron_gofberg)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 10:19pm UTC](https://discuss.elastic.co/t/streaming-binary-filess-decoded-data-to-elastic/212712 "2019-12-20T22:19:38Z")
I am wondering what tools I can use for streaming decoded data from binary files to elastic the the fastest way. files are located at library and i want the library to be continuously checked for new files so the new da… --- ## [Filebeat not collecting docker logs](https://discuss.elastic.co/t/filebeat-not-collecting-docker-logs/212568)
**Author:** [@algallagher](https://discuss.elastic.co/u/algallagher)\ **Replies:** 4\ **Last updated:** [December 20, 2019, 7:29pm UTC](https://discuss.elastic.co/t/filebeat-not-collecting-docker-logs/212568 "2019-12-20T19:29:28Z")
I am running an ELK stack as 3 separate containers running locally (kibana, logstash, elasticsearch). I am running Docker Desktop for Windows (though I plan to migrate this entire setup to AWS). I am also running a java… --- ## [Searching custom fields does not work with Full Text query mode(?)](https://discuss.elastic.co/t/searching-custom-fields-does-not-work-with-full-text-query-mode/212683)
**Author:** [@john\_eapen](https://discuss.elastic.co/u/john_eapen)\ **Replies:** 1\ **Last updated:** [December 20, 2019, 5:47pm UTC](https://discuss.elastic.co/t/searching-custom-fields-does-not-work-with-full-text-query-mode/212683 "2019-12-20T17:47:22Z")
Hi My log data contains several fields that Filebeat doesn’t know about in advance. ( eg: myapp.traceId, myapp.host.name etc. ). To avoid conflicts these are not directly root but under a root property "myapp". Now, wh… --- ## [Would dropping events by regexp match be possible?](https://discuss.elastic.co/t/would-dropping-events-by-regexp-match-be-possible/212642)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 2\ **Last updated:** [December 20, 2019, 5:29pm UTC](https://discuss.elastic.co/t/would-dropping-events-by-regexp-match-be-possible/212642 "2019-12-20T17:29:32Z")
Any suggestions on how we best would target just below events from our backup client for the purpose of dropping such in winlogbeat? Could we do a regexp match on event\_data.ProcessName as event.code or event.action mig… --- ## [No Kibana Dashboards in Saved Objects/Dashboards with correct metric beat settings](https://discuss.elastic.co/t/no-kibana-dashboards-in-saved-objects-dashboards-with-correct-metric-beat-settings/212567)
**Author:** [@cgutshall](https://discuss.elastic.co/u/cgutshall)\ **Replies:** 4\ **Last updated:** [December 20, 2019, 4:08pm UTC](https://discuss.elastic.co/t/no-kibana-dashboards-in-saved-objects-dashboards-with-correct-metric-beat-settings/212567 "2019-12-20T16:08:18Z")
I have a single node EC2 Instance running both Elasticsearch and Kibana, which I am sending kubernetes metrics to from a daemonset running a metricbeat pod for each node In the cluster. Even with the following settings … --- ## [Mapping for Elasticsearch \[easy\]](https://discuss.elastic.co/t/mapping-for-elasticsearch-easy/212036)
**Author:** [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Replies:** 1\ **Last updated:** [December 20, 2019, 12:25pm UTC](https://discuss.elastic.co/t/mapping-for-elasticsearch-easy/212036 "2019-12-20T12:25:03Z")
I've got an easy question. I've got this file: # epoch, metric1, metric2, metric3 1576425930,0.0718,0.0127,1 How can I tell Filebeat to send it to Elasticsearch and use the correct mapping. My config file currently l… --- ## [Services Down & % of Services Down](https://discuss.elastic.co/t/services-down-of-services-down/212660)
**Author:** [@ginu](https://discuss.elastic.co/u/ginu)\ **Replies:** 0\ **Last updated:** [December 20, 2019, 12:18pm UTC](https://discuss.elastic.co/t/services-down-of-services-down/212660 "2019-12-20T12:18:03Z")
Hi All I have a metricbeat which checks the service status which monitors the WWWServices. See below the Query event.dataset:windows.service AND windows.service.name: W3SVC AND windows.service.start\_type.keyword:"Autom… --- ## [\[METRICBEAT\] Infrastructure kubernetes inventory empty](https://discuss.elastic.co/t/metricbeat-infrastructure-kubernetes-inventory-empty/209054)
**Author:** [@Benjamin\_Carriou](https://discuss.elastic.co/u/Benjamin_Carriou)\ **Replies:** 5\ **Last updated:** [December 20, 2019, 11:32am UTC](https://discuss.elastic.co/t/metricbeat-infrastructure-kubernetes-inventory-empty/209054 "2019-12-20T11:32:21Z")
Hi, I have install metricbeats on my k8s cluster and I can see hosts metrics in infrastructure inventory module but kubernetes metrics (pods) are empty: We are some persons who this issue appears: https://discuss.e… --- ## [Fielddata is disabled on text fields by default. Set fielddata=true on \[host.name\] in order to load fielddata in memory by uninverting the inverted index](https://discuss.elastic.co/t/fielddata-is-disabled-on-text-fields-by-default-set-fielddata-true-on-host-name-in-order-to-load-fielddata-in-memory-by-uninverting-the-inverted-index/212111)
**Author:** [@skdasari](https://discuss.elastic.co/u/skdasari)\ **Replies:** 4\ **Last updated:** [December 20, 2019, 8:57am UTC](https://discuss.elastic.co/t/fielddata-is-disabled-on-text-fields-by-default-set-fielddata-true-on-host-name-in-order-to-load-fielddata-in-memory-by-uninverting-the-inverted-index/212111 "2019-12-20T08:57:26Z")
Hello, Something weird happend lately in my elasticsearch setup letely. Metricbeat index got replaced with a new name like the old one is like metricbeat.x.date, But now the index is showing as metricbeat .version and w… --- ## [Helm chart for Auditbeat](https://discuss.elastic.co/t/helm-chart-for-auditbeat/212559)
**Author:** [@KenRider](https://discuss.elastic.co/u/KenRider)\ **Replies:** 1\ **Last updated:** [December 20, 2019, 6:35am UTC](https://discuss.elastic.co/t/helm-chart-for-auditbeat/212559 "2019-12-20T06:35:04Z")
Is there going to be a Helm chart available for Auditbeat any time soon? --- ## [Setup.template.name not working with ILM (?)](https://discuss.elastic.co/t/setup-template-name-not-working-with-ilm/212562)
**Author:** [@john\_eapen](https://discuss.elastic.co/u/john_eapen)\ **Replies:** 1\ **Last updated:** [December 20, 2019, 2:51am UTC](https://discuss.elastic.co/t/setup-template-name-not-working-with-ilm/212562 "2019-12-20T02:51:10Z")
Hi, I have slightly confused about some of the posts I have seen on this. Does custom template not work when ILM is enabled in filebeat ? (Needless to say,I need both these features.) I hv specified my ES index as: … --- ## [Cisco Module - Cannot Query Event.Original field](https://discuss.elastic.co/t/cisco-module-cannot-query-event-original-field/212433)
**Author:** [@jameswatson3](https://discuss.elastic.co/u/jameswatson3)\ **Replies:** 1\ **Last updated:** [December 19, 2019, 8:24pm UTC](https://discuss.elastic.co/t/cisco-module-cannot-query-event-original-field/212433 "2019-12-19T20:24:40Z")
The filebeat 7.5 Cisco module is successfully sending asa, ftd and ios documents to elasticsearch. Using the pipelines created by filebeat modules cisco --pipelines, the documents are being parsed nicely and stats are … [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=288) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=290)