# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=89 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 90 --- ## [How to read the monitoring log of filebeat?](https://discuss.elastic.co/t/how-to-read-the-monitoring-log-of-filebeat/305721)
**Author:** [@Kuo\_Hugo](https://discuss.elastic.co/u/Kuo_Hugo)\ **Replies:** 2\ **Last updated:** [May 31, 2022, 3:24am UTC](https://discuss.elastic.co/t/how-to-read-the-monitoring-log-of-filebeat/305721 "2022-05-31T03:24:40Z")
I'm working on tuning for the filebeat to process logs. Changing the output.Elasticsearch There're 7 Elasticsearch targets in the hosts list. workers:2 bulk\_max\_size: 2048 flush\_interval: 5 It's around 600 log line… --- ## [Kubernetes filebeat best practices](https://discuss.elastic.co/t/kubernetes-filebeat-best-practices/305270)
**Author:** [@lbchv](https://discuss.elastic.co/u/lbchv)\ **Replies:** 1\ **Last updated:** [May 31, 2022, 2:31am UTC](https://discuss.elastic.co/t/kubernetes-filebeat-best-practices/305270 "2022-05-31T02:31:55Z")
The documentation says to avoid harvesting symlinks when dialing with file rotation (link): When dealing with file rotation, avoid harvesting symlinks. Instead use the paths setting to point to the original file, and s… --- ## [Cannot able to pull RDS metrics using metricbeat](https://discuss.elastic.co/t/cannot-able-to-pull-rds-metrics-using-metricbeat/305717)
**Author:** [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Replies:** 2\ **Last updated:** [May 30, 2022, 8:08pm UTC](https://discuss.elastic.co/t/cannot-able-to-pull-rds-metrics-using-metricbeat/305717 "2022-05-30T20:08:40Z")
Hi Team, I am trying to pull the AWS RDS metrics from metricbeat using AWS module and i have got the arn\_role to pull the metrics. But when i run metricbeat i am getting below error. Could you please help me to solve th… --- ## [Unable to uninstall elastic-agent](https://discuss.elastic.co/t/unable-to-uninstall-elastic-agent/305347)
**Author:** [@liwenbo97](https://discuss.elastic.co/u/liwenbo97)\ **Replies:** 1\ **Last updated:** [May 30, 2022, 4:43pm UTC](https://discuss.elastic.co/t/unable-to-uninstall-elastic-agent/305347 "2022-05-30T16:43:43Z")
n227-012-170 ➜ ~ sudo elastic-agent uninstall Error: can only be uninstall by executing the installed Elastic Agent at: /usr/bin/elastic-agent For help, please see our troubleshooting guide at Troubleshoot common probl… --- ## [Filebeat 8.2.1 kubernetes autodiscover template with multiple conditions doesn't work](https://discuss.elastic.co/t/filebeat-8-2-1-kubernetes-autodiscover-template-with-multiple-conditions-doesnt-work/305693)
**Author:** [@codrut\_alexandru](https://discuss.elastic.co/u/codrut_alexandru)\ **Replies:** 1\ **Last updated:** [May 30, 2022, 3:16pm UTC](https://discuss.elastic.co/t/filebeat-8-2-1-kubernetes-autodiscover-template-with-multiple-conditions-doesnt-work/305693 "2022-05-30T15:16:12Z")
Hi, Configuring filebeat version 8.2.1 autodiscover template with multiple conditions doesn't work: This works: templates: - condition.or: - not.equals: … --- ## [Can ElasticAgent/filebeat ingest a log/filestream input metadata as document data?](https://discuss.elastic.co/t/can-elasticagent-filebeat-ingest-a-log-filestream-input-metadata-as-document-data/305734)
**Author:** [@bunjamins](https://discuss.elastic.co/u/bunjamins)\ **Replies:** 1\ **Last updated:** [May 30, 2022, 3:12pm UTC](https://discuss.elastic.co/t/can-elasticagent-filebeat-ingest-a-log-filestream-input-metadata-as-document-data/305734 "2022-05-30T15:12:35Z")
specifically we have a legacy app that doesn't log a timestamp but logs to new files, is it possible to read file ctime as @timestamp? --- ## [Filebeat unable to recognize the colorful file content](https://discuss.elastic.co/t/filebeat-unable-to-recognize-the-colorful-file-content/305921)
**Author:** [@sujeetkp](https://discuss.elastic.co/u/sujeetkp)\ **Replies:** 1\ **Last updated:** [May 30, 2022, 2:54pm UTC](https://discuss.elastic.co/t/filebeat-unable-to-recognize-the-colorful-file-content/305921 "2022-05-30T14:54:29Z")
I am trying to read a log file using filebeat. The log file looks as below. When filebeat reads it, it is displaying some special characters as below. Looks like filebeat is not able to recognize/ignore the color… --- ## [If conditions when dividing logs?](https://discuss.elastic.co/t/if-conditions-when-dividing-logs/305497)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 8\ **Last updated:** [May 30, 2022, 7:52am UTC](https://discuss.elastic.co/t/if-conditions-when-dividing-logs/305497 "2022-05-30T07:52:23Z")
Hi, I separate the logs with this config in filebeat - type: log enabled: true paths: - D:\\elastic\_stack\\LOGS\\CMS\_LOGS\\\* fields: kafka\_topic: "kafka-topic-cms-test1" tags: \["CMS"\] multiline.type: … --- ## [Kafka + Metricbeat error:Failed to fetch offset for (topic, partition)](https://discuss.elastic.co/t/kafka-metricbeat-error-failed-to-fetch-offset-for-topic-partition/305777)
**Author:** [@ekeras](https://discuss.elastic.co/u/ekeras)\ **Replies:** 0\ **Last updated:** [May 27, 2022, 8:15am UTC](https://discuss.elastic.co/t/kafka-metricbeat-error-failed-to-fetch-offset-for-topic-partition/305777 "2022-05-27T08:15:08Z")
I am running elastic stack by this repository. I just changed the network elastic stack is being hosted on, so it would be on the same network as Kafka is. To set up monitoring kafka metrics with Elastic, I have followe… --- ## [Metricbeat 8.2.1 monitoring Elasticsearch 8.2.1 - Template issue](https://discuss.elastic.co/t/metricbeat-8-2-1-monitoring-elasticsearch-8-2-1-template-issue/305649)
**Author:** [@bar0n36](https://discuss.elastic.co/u/bar0n36)\ **Replies:** 5\ **Last updated:** [May 26, 2022, 10:07pm UTC](https://discuss.elastic.co/t/metricbeat-8-2-1-monitoring-elasticsearch-8-2-1-template-issue/305649 "2022-05-26T22:07:40Z")
There appears to be a couple of issues with the Metricbeat 8.2.1 template for the Elasticsearch metric monitoring. I am running a newly built 8.2.1 cluster (in Docker) and attempting to use Metricbeat to gather Stack Mon… --- ## [Too many logs for filebeat?](https://discuss.elastic.co/t/too-many-logs-for-filebeat/305486)
**Author:** [@seePyou](https://discuss.elastic.co/u/seePyou)\ **Replies:** 3\ **Last updated:** [May 26, 2022, 5:59pm UTC](https://discuss.elastic.co/t/too-many-logs-for-filebeat/305486 "2022-05-26T17:59:27Z")
I'm trying to run a POC for Filebeats on capturing logs from Signiant file delivery. We get files in and we send files out, and there are processing steps in Signiant. For each event, for every process, there is a log fi… --- ## [Filebeat raise error:filestream input id already exist](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241)
**Author:** [@duduniao](https://discuss.elastic.co/u/duduniao)\ **Replies:** 8\ **Last updated:** [May 26, 2022, 4:15pm UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241 "2022-05-26T16:15:01Z")
I install filebeat-8.1.2 on centos 7, then use filestream input to read log files: - type: filestream id: ro-server-run-log paths: \['/data/ro\_jplive/log/\*-\*-\*-????-??-??-??.log'\] prospector.scanner.exclude\_files:… --- ## [ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://elasticsearch:9200)): 401 Unauthorized](https://discuss.elastic.co/t/error-pipeline-output-go-100-failed-to-connect-to-backoff-elasticsearch-http-elasticsearch-9200-401-unauthorized/305687)
**Author:** [@kosmylo](https://discuss.elastic.co/u/kosmylo)\ **Replies:** 2\ **Last updated:** [May 26, 2022, 11:13am UTC](https://discuss.elastic.co/t/error-pipeline-output-go-100-failed-to-connect-to-backoff-elasticsearch-http-elasticsearch-9200-401-unauthorized/305687 "2022-05-26T11:13:39Z")
I have an express app with nginx that writes the web access logs through filebeat/logstash/Elasticsearch/kibana. I wanted to add packetbeat to this pipeline. The docker-compose is the following: packetbeat: build: … --- ## [Monitoring svclog using filebeat](https://discuss.elastic.co/t/monitoring-svclog-using-filebeat/305534)
**Author:** [@Gaurav\_kr](https://discuss.elastic.co/u/Gaurav_kr)\ **Replies:** 2\ **Last updated:** [May 26, 2022, 7:07am UTC](https://discuss.elastic.co/t/monitoring-svclog-using-filebeat/305534 "2022-05-26T07:07:25Z")
Hi All, I am trying to find a way to monitor .svclog using filebeat. I can get the log but having some trouble like: Unable to harvest svclog which has large log line not sure if we have any command to make sure all … --- ## [Metricbeat error: missing field accessing 'metricbeat.modules.21.hosts.0](https://discuss.elastic.co/t/metricbeat-error-missing-field-accessing-metricbeat-modules-21-hosts-0/305648)
**Author:** [@alexander2](https://discuss.elastic.co/u/alexander2)\ **Replies:** 0\ **Last updated:** [May 26, 2022, 2:15am UTC](https://discuss.elastic.co/t/metricbeat-error-missing-field-accessing-metricbeat-modules-21-hosts-0/305648 "2022-05-26T02:15:28Z")
Hello guys. i installed metricbeat on my centos 8 machine. with this configuration file. ########################## Metricbeat Configuration ########################### # This file is a full configuration example docu… --- ## [Mapper \[user\_agent.version\] cannot be changed from type \[text\] to \[keyword\] How to disable this warining?](https://discuss.elastic.co/t/mapper-user-agent-version-cannot-be-changed-from-type-text-to-keyword-how-to-disable-this-warining/305638)
**Author:** [@Sebastian\_Pacuk](https://discuss.elastic.co/u/Sebastian_Pacuk)\ **Replies:** 0\ **Last updated:** [May 25, 2022, 7:09pm UTC](https://discuss.elastic.co/t/mapper-user-agent-version-cannot-be-changed-from-type-text-to-keyword-how-to-disable-this-warining/305638 "2022-05-25T19:09:23Z")
I'm newbie to elasticsearch. What I want to do is to gather logs with filebeat from kubernetes cluster and send them to elasticsearch. My filebeat config file looks like this: \`filebeat.yml: | filebeat.inputs: type: … --- ## [AWS integration with Elasticsearch](https://discuss.elastic.co/t/aws-integration-with-elasticsearch/305284)
**Author:** [@Atul\_Chadha](https://discuss.elastic.co/u/Atul_Chadha)\ **Replies:** 5\ **Last updated:** [May 25, 2022, 1:52pm UTC](https://discuss.elastic.co/t/aws-integration-with-elasticsearch/305284 "2022-05-25T13:52:48Z")
I am trying to setup AWS integration with Elasticsearch 7.17.3 and have had some success. I am not able to get data for few services and the filebeat.json log shows the below error. I am seeing the region code twice in … --- ## [Filebeat daemonset losing messages](https://discuss.elastic.co/t/filebeat-daemonset-losing-messages/305515)
**Author:** [@Mike\_Williams](https://discuss.elastic.co/u/Mike_Williams)\ **Replies:** 1\ **Last updated:** [May 25, 2022, 10:43am UTC](https://discuss.elastic.co/t/filebeat-daemonset-losing-messages/305515 "2022-05-25T10:43:22Z")
Hi, Roughly this question has been asked a few times over the years but I've yet to find any real solutions. We've got filebeat deployed as a daemonset in an on-prem k8s cluster. The config is very simple. filebeat.a… --- ## [Filebeat not read log, log msg "File didn't change"](https://discuss.elastic.co/t/filebeat-not-read-log-log-msg-file-didnt-change/305545)
**Author:** [@pi314](https://discuss.elastic.co/u/pi314)\ **Replies:** 1\ **Last updated:** [May 25, 2022, 2:37am UTC](https://discuss.elastic.co/t/filebeat-not-read-log-log-msg-file-didnt-change/305545 "2022-05-25T02:37:50Z")
Hi, I have configured filebeat to send a file to elastic but it is not happening, for some reason which I can not discover is not reading the file. This is my config: logging.level: debug logging.metrics.period: 1m log… --- ## [Error fetching data for metricset linux.pageinfo: error reading pagetypeinfo](https://discuss.elastic.co/t/error-fetching-data-for-metricset-linux-pageinfo-error-reading-pagetypeinfo/305104)
**Author:** [@jplindquist](https://discuss.elastic.co/u/jplindquist)\ **Replies:** 1\ **Last updated:** [May 24, 2022, 6:07pm UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-linux-pageinfo-error-reading-pagetypeinfo/305104 "2022-05-24T18:07:13Z")
We are using metricbeat 7.17.3 as of this post, and are receiving an error on several systems which are configured to use the linux.pageinfo metricset, also reported here, but has since auto closed. May 17 15:38:58 exam… --- ## [Mapping conflict with host.ip in filebeat](https://discuss.elastic.co/t/mapping-conflict-with-host-ip-in-filebeat/305502)
**Author:** [@PJ\_ceSofnI\_tseT](https://discuss.elastic.co/u/PJ_ceSofnI_tseT)\ **Replies:** 0\ **Last updated:** [May 24, 2022, 12:19pm UTC](https://discuss.elastic.co/t/mapping-conflict-with-host-ip-in-filebeat/305502 "2022-05-24T12:19:11Z")
Hi all, I am using filebeat on different hosts but on one of them a conflict has appeared for the host.ip field. This field appears to have the types: ip and text. This happens because both the IPv4 and IPv6 version… --- ## [Help me in configure filebeat index](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253)
**Author:** [@Rajesh119](https://discuss.elastic.co/u/Rajesh119)\ **Replies:** 21\ **Last updated:** [May 24, 2022, 11:58am UTC](https://discuss.elastic.co/t/help-me-in-configure-filebeat-index/305253 "2022-05-24T11:58:34Z")
i use this command, it will not shown my filebeat index curl http://localhost:9200/\_cat/indices?v And i use below command to know the status filebeat -e -c /etc/filebeat/filebeat.yml output: {"log.level":"info",… --- ## [Filebeat doesn't send logs to elasticsearch](https://discuss.elastic.co/t/filebeat-doesnt-send-logs-to-elasticsearch/305301)
**Author:** [@elduderino](https://discuss.elastic.co/u/elduderino)\ **Replies:** 2\ **Last updated:** [May 24, 2022, 11:31am UTC](https://discuss.elastic.co/t/filebeat-doesnt-send-logs-to-elasticsearch/305301 "2022-05-24T11:31:08Z")
I got some logs and filebeat was shipping them to the ES server just fine but all of a sudden seems to be stopped shipping some files in different folders in the same machine. Filebeat service is running. Did anyone expe… --- ## [Expand\_keys doesn't work if target option is used in decode\_json\_fields beat processor](https://discuss.elastic.co/t/expand-keys-doesnt-work-if-target-option-is-used-in-decode-json-fields-beat-processor/305276)
**Author:** [@iorfix](https://discuss.elastic.co/u/iorfix)\ **Replies:** 3\ **Last updated:** [May 24, 2022, 10:35am UTC](https://discuss.elastic.co/t/expand-keys-doesnt-work-if-target-option-is-used-in-decode-json-fields-beat-processor/305276 "2022-05-24T10:35:08Z")
In decode\_json\_fields processor, expanded notation works only if target is left to "". In fact, in source code ( beats/decode\_json\_fields.go at master · elastic/beats · GitHub ) there is a clear if section: if target… --- ## [Running multiple filebeat instances](https://discuss.elastic.co/t/running-multiple-filebeat-instances/305423)
**Author:** [@justplanenutz](https://discuss.elastic.co/u/justplanenutz)\ **Replies:** 2\ **Last updated:** [May 23, 2022, 8:59pm UTC](https://discuss.elastic.co/t/running-multiple-filebeat-instances/305423 "2022-05-23T20:59:07Z")
We have a system that is very chatty with regard to the number of logs produced. Our current production implementation has a single filebeat container ( in k8s ) running and it is struggling to keep up. We have adjusted … --- ## [Environment name instead of hostname](https://discuss.elastic.co/t/environment-name-instead-of-hostname/305359)
**Author:** [@Anagha\_nambiar](https://discuss.elastic.co/u/Anagha_nambiar)\ **Replies:** 4\ **Last updated:** [May 23, 2022, 4:29pm UTC](https://discuss.elastic.co/t/environment-name-instead-of-hostname/305359 "2022-05-23T16:29:15Z")
Hi All, Suppose I have installed metricbeat in 3 servers and their machine names are IIS1, IIS2 and IIS3. If I have to filter metric data for dashboard creation in Kibana, then I used the query "agent.hostname"or "host… --- ## [MetricBeat no longer creating new indexes](https://discuss.elastic.co/t/metricbeat-no-longer-creating-new-indexes/305400)
**Author:** [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Replies:** 2\ **Last updated:** [May 23, 2022, 3:39pm UTC](https://discuss.elastic.co/t/metricbeat-no-longer-creating-new-indexes/305400 "2022-05-23T15:39:48Z")
Hello. I've been using MetricBeat for a while now and it was working just fine. All of a sudden, about mid April, I noticed that it stopped logging to the index. Upon closer inspection, all of the metricbeat agents ar… --- ## [Elastic agent not shipping syslog](https://discuss.elastic.co/t/elastic-agent-not-shipping-syslog/305286)
**Author:** [@Thomas\_Thorburn](https://discuss.elastic.co/u/Thomas_Thorburn)\ **Replies:** 1\ **Last updated:** [May 23, 2022, 2:17pm UTC](https://discuss.elastic.co/t/elastic-agent-not-shipping-syslog/305286 "2022-05-23T14:17:00Z")
Hi all, I'm using elastic agent v8.2.0 being sent to an Elasticsearch service cluster on v8.2.0 using the recommended tar install method. The agents are healthy with the system integration enabled and ship logs from /va… --- ## [Beats in docker after 7.17](https://discuss.elastic.co/t/beats-in-docker-after-7-17/302429)
**Author:** [@Yustas](https://discuss.elastic.co/u/Yustas)\ **Replies:** 6\ **Last updated:** [May 23, 2022, 8:03am UTC](https://discuss.elastic.co/t/beats-in-docker-after-7-17/302429 "2022-05-23T08:03:18Z")
Hello Found problem with dockers image of beats from version 7.17 and higher. Early images was near 600 Mb size After 7.17 image baset on lightweight image near 300 Mb But in new images: no tzdate, so cant use -e T… --- ## [Issues with Filebeat, how to debug?](https://discuss.elastic.co/t/issues-with-filebeat-how-to-debug/304754)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 4\ **Last updated:** [May 22, 2022, 4:03am UTC](https://discuss.elastic.co/t/issues-with-filebeat-how-to-debug/304754 "2022-05-22T04:03:18Z")
Hi Team, I am facing an weird issue with Filebeat dumping data in Elasticsearch. My filebeat has suddenly stopped the data dumping to Elasticsearch. If we restart the service it starts dumping the data momentarily but s… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=88) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=90)