# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=407 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 408 --- ## [Exclude\_lines per file](https://discuss.elastic.co/t/exclude-lines-per-file/152129)
**Author:** [@Bkt](https://discuss.elastic.co/u/Bkt)\ **Replies:** 4\ **Last updated:** [November 5, 2018, 4:41pm UTC](https://discuss.elastic.co/t/exclude-lines-per-file/152129 "2018-11-05T16:41:45Z")
Hello, I have a filebeat which has 2 files, I would like to have exclude lines different for each one, is it possible to add exclude\_lines per path? the potential config is as below i.e. for access.log I want to exclud… --- ## [FileBeats on Solaris 11.3](https://discuss.elastic.co/t/filebeats-on-solaris-11-3/155392)
**Author:** [@sarban.kumar](https://discuss.elastic.co/u/sarban.kumar)\ **Replies:** 3\ **Last updated:** [November 5, 2018, 2:42pm UTC](https://discuss.elastic.co/t/filebeats-on-solaris-11-3/155392 "2018-11-05T14:42:59Z")
Hi, I 've to ship log through Filebeats , its oracle EBS where weblogic is the core components. Looking for Filebeat installation on Solaris 11.3 SPARC system, came through below link , which says TOPBEAT does not supp… --- ## [Filebeat inside Kubernetes setup questions](https://discuss.elastic.co/t/filebeat-inside-kubernetes-setup-questions/155101)
**Author:** [@rocketraman](https://discuss.elastic.co/u/rocketraman)\ **Replies:** 2\ **Last updated:** [November 5, 2018, 2:02pm UTC](https://discuss.elastic.co/t/filebeat-inside-kubernetes-setup-questions/155101 "2018-11-05T14:02:23Z")
I'm trying to use hints-based autodiscover with Filebeat 6.4.2 inside Kubernetes. The basic setup is working (through some trials and tribulations noted in earlier messages in this forum). However, now I'm trying to ena… --- ## [Packet loss in af\_packetbeat mode](https://discuss.elastic.co/t/packet-loss-in-af-packetbeat-mode/154186)
**Author:** [@Ezy2015](https://discuss.elastic.co/u/Ezy2015)\ **Replies:** 4\ **Last updated:** [November 5, 2018, 8:42am UTC](https://discuss.elastic.co/t/packet-loss-in-af-packetbeat-mode/154186 "2018-11-05T08:42:23Z")
Packetbeat Version: 6.2.4 Operating System: ubuntu16.04 Hello, I found packetbeat lost packets when capture mirrored HTTP traffic.The way I found is that I send 100 packets every 10 minutes, and then count how many pac… --- ## [Filebeats for UFW logs - how to define fields?](https://discuss.elastic.co/t/filebeats-for-ufw-logs-how-to-define-fields/155263)
**Author:** [@bvickers](https://discuss.elastic.co/u/bvickers)\ **Replies:** 1\ **Last updated:** [November 5, 2018, 8:52am UTC](https://discuss.elastic.co/t/filebeats-for-ufw-logs-how-to-define-fields/155263 "2018-11-05T08:52:17Z")
Hello. I am trying to set up Elastic Stack to manage some typical and some custom logs. I have Windows and Linux servers to grab logs from and I decided to use Beats to make it a little bit more secure (using logstash … --- ## [Require clarification on filebeat ingest pipeline](https://discuss.elastic.co/t/require-clarification-on-filebeat-ingest-pipeline/155346)
**Author:** [@Nithani25](https://discuss.elastic.co/u/Nithani25)\ **Replies:** 1\ **Last updated:** [November 5, 2018, 8:41am UTC](https://discuss.elastic.co/t/require-clarification-on-filebeat-ingest-pipeline/155346 "2018-11-05T08:41:26Z")
Hi Team, I have installed a brand new setup in my production environment, i am successful in injecting the apache access and error logs via filebeat apache2 module. however, my access log seems to have some additonal da… --- ## [How to remove beat field and host field](https://discuss.elastic.co/t/how-to-remove-beat-field-and-host-field/155334)
**Author:** [@luzhzhsoft](https://discuss.elastic.co/u/luzhzhsoft)\ **Replies:** 2\ **Last updated:** [November 5, 2018, 6:03am UTC](https://discuss.elastic.co/t/how-to-remove-beat-field-and-host-field/155334 "2018-11-05T06:03:20Z")
My output contains beat ，host ，@timestamp @metadat filed as default.I want remove beat ,host fields from input.I define processor ,but it not work for beat and host filelds.I do not want to use pipeline ,because I hav… --- ## [Is there any workaround for Filebeat max\_retries since it never works now](https://discuss.elastic.co/t/is-there-any-workaround-for-filebeat-max-retries-since-it-never-works-now/155342)
**Author:** [@guanghaofan](https://discuss.elastic.co/u/guanghaofan)\ **Replies:** 1\ **Last updated:** [November 5, 2018, 6:02am UTC](https://discuss.elastic.co/t/is-there-any-workaround-for-filebeat-max-retries-since-it-never-works-now/155342 "2018-11-05T06:02:04Z")
hi experts, now my case is I want to set the max\_reties for bulk send request, then filebeat can smoothly consume the new files and won't be blocked by the bulk send failures... see my issue in this link: https://discu… --- ## [How to use filebeat and logstash for different log](https://discuss.elastic.co/t/how-to-use-filebeat-and-logstash-for-different-log/154538)
**Author:** [@aabababba](https://discuss.elastic.co/u/aabababba)\ **Replies:** 8\ **Last updated:** [November 5, 2018, 5:34am UTC](https://discuss.elastic.co/t/how-to-use-filebeat-and-logstash-for-different-log/154538 "2018-11-05T05:34:47Z")
1 /var/log/message Oct 30 09:04:35 ci04 dbus-daemon: dbus\[616\]: \[system\] Successfully activated service 'org.freedesktop.problems' 2 /var/log/tomcat/catalina.out \[INFO \] \[2018-10-29 16:44:30,945\] \[DubboServerHandler… --- ## [How to read Metricbeat iowait time](https://discuss.elastic.co/t/how-to-read-metricbeat-iowait-time/155311)
**Author:** [@grantcurell](https://discuss.elastic.co/u/grantcurell)\ **Replies:** 0\ **Last updated:** [November 4, 2018, 6:32pm UTC](https://discuss.elastic.co/t/how-to-read-metricbeat-iowait-time/155311 "2018-11-04T18:32:49Z")
I'm diagnosing some performance issues with a high speed security sensor I'm running. I strongly suggest disk I/O is my culprit. Metricbeat's has really helped me get insight on my stack's performance, but I'm not actu… --- ## [How to ignore some index](https://discuss.elastic.co/t/how-to-ignore-some-index/154980)
**Author:** [@mhsankar](https://discuss.elastic.co/u/mhsankar)\ **Replies:** 2\ **Last updated:** [November 4, 2018, 7:10am UTC](https://discuss.elastic.co/t/how-to-ignore-some-index/154980 "2018-11-04T07:10:40Z")
Hi im using Packetbeat to log my request. it log all elastic request. i want to ignore all index packetbeat-\* . how can i do it? --- ## [Filebeat Inputs: Docker vs Log with options](https://discuss.elastic.co/t/filebeat-inputs-docker-vs-log-with-options/155122)
**Author:** [@justinw](https://discuss.elastic.co/u/justinw)\ **Replies:** 2\ **Last updated:** [November 4, 2018, 1:43am UTC](https://discuss.elastic.co/t/filebeat-inputs-docker-vs-log-with-options/155122 "2018-11-04T01:43:03Z")
Out of curiosity, what's the difference between the two: - type: log paths: - /var/lib/docker/containers/\*/\*-json.log json.keys\_under\_root: true json.add\_error\_key: true json.message\_key: log ## assume downs… --- ## [Error calculating CPU time change for docker (filebeat 6.4.2)](https://discuss.elastic.co/t/error-calculating-cpu-time-change-for-docker-filebeat-6-4-2/155279)
**Author:** [@justinw](https://discuss.elastic.co/u/justinw)\ **Replies:** 0\ **Last updated:** [November 4, 2018, 1:32am UTC](https://discuss.elastic.co/t/error-calculating-cpu-time-change-for-docker-filebeat-6-4-2/155279 "2018-11-04T01:32:18Z")
Hi, Seeing some errors on metricbeat v6.4.2, on collection and sending of CPU metrics. My configuration is largely the same as what is provided here, other than swapping out appropriate environment config. https://www.… --- ## [How to install the awsbeat](https://discuss.elastic.co/t/how-to-install-the-awsbeat/155259)
**Author:** [@Shahid\_Chaudhary](https://discuss.elastic.co/u/Shahid_Chaudhary)\ **Replies:** 2\ **Last updated:** [November 3, 2018, 3:09pm UTC](https://discuss.elastic.co/t/how-to-install-the-awsbeat/155259 "2018-11-03T15:09:26Z")
Hi Dear Tech Team, I need to get connected with my AWS account and get my infrastructure logs from AWS to ELK stack. I think so the AWSBeat can help me on this to get required logs. I need you guys help and to install … --- ## [Structured logging (json) via Kubernetes hinted autodiscover](https://discuss.elastic.co/t/structured-logging-json-via-kubernetes-hinted-autodiscover/155118)
**Author:** [@rocketraman](https://discuss.elastic.co/u/rocketraman)\ **Replies:** 1\ **Last updated:** [November 3, 2018, 1:58pm UTC](https://discuss.elastic.co/t/structured-logging-json-via-kubernetes-hinted-autodiscover/155118 "2018-11-03T13:58:33Z")
I am using autodiscovery with hints in Kubernetes. Many of my containers log with single JSON per-line structured logging as described @ https://www.elastic.co/blog/structured-logging-filebeat. However, looking at the … --- ## [Error data metricbeat kubernetes](https://discuss.elastic.co/t/error-data-metricbeat-kubernetes/155244)
**Author:** [@Edgard\_Andres\_Piment](https://discuss.elastic.co/u/Edgard_Andres_Piment)\ **Replies:** 1\ **Last updated:** [November 3, 2018, 1:27pm UTC](https://discuss.elastic.co/t/error-data-metricbeat-kubernetes/155244 "2018-11-03T13:27:33Z")
Dears friends, I have a problem with metricbeat in k8s on premise, well the problem is the next: elastichsearch "metricbeat":{"kubernetes":{"container":{"events":3,"failures":3},"node":{"events":3,"failures":3},"pod"… --- ## [Filebeat monitoring problem](https://discuss.elastic.co/t/filebeat-monitoring-problem/154278)
**Author:** [@pauldon2](https://discuss.elastic.co/u/pauldon2)\ **Replies:** 6\ **Last updated:** [November 3, 2018, 9:03am UTC](https://discuss.elastic.co/t/filebeat-monitoring-problem/154278 "2018-11-03T09:03:20Z")
I tried to enable filebeat monitoring on elasticsearch. Enabled it in the configuration: xpack.monitoring: enabled: true elasticsearch: hosts: \["http://10.10.8.77:9200"\] Thus configured 8 servers. But in kibana I s… --- ## [Failed to perform any bulk index operations: 403 Forbidden:](https://discuss.elastic.co/t/failed-to-perform-any-bulk-index-operations-403-forbidden/153207)
**Author:** [@sirababu](https://discuss.elastic.co/u/sirababu)\ **Replies:** 11\ **Last updated:** [November 2, 2018, 11:44pm UTC](https://discuss.elastic.co/t/failed-to-perform-any-bulk-index-operations-403-forbidden/153207 "2018-11-02T23:44:00Z")
Hello Search Guru's I am getting this error, i have a AWS ELK POC cluster(1 node), i am using filebeat to ingest data, getting this error With curl i can create index, any suggestions, thanks elasticsearch/client.go:3… --- ## [Filebeat high memory usage v6.4.2](https://discuss.elastic.co/t/filebeat-high-memory-usage-v6-4-2/155186)
**Author:** [@grantk](https://discuss.elastic.co/u/grantk)\ **Replies:** 6\ **Last updated:** [November 2, 2018, 9:57pm UTC](https://discuss.elastic.co/t/filebeat-high-memory-usage-v6-4-2/155186 "2018-11-02T21:57:56Z")
I am running a test using filebeat and 5 json log files each writing about 200 entries/second into its own json log file. I have not changed the defaults in filebeat other than the json entries: json.message\_key: level … --- ## [Filebeat is not sending log entries/logs to logstash server](https://discuss.elastic.co/t/filebeat-is-not-sending-log-entries-logs-to-logstash-server/155227)
**Author:** [@HRG](https://discuss.elastic.co/u/HRG)\ **Replies:** 1\ **Last updated:** [November 2, 2018, 8:29pm UTC](https://discuss.elastic.co/t/filebeat-is-not-sending-log-entries-logs-to-logstash-server/155227 "2018-11-02T20:29:09Z")
I have setup filebeat on 3 servers. The filebeat is able to send logs to logstash server without issues from 2 servers. From 3rd server, i can see file beat is running without any issues and logstash also running on lo… --- ## [Elasticsearch permissions required by Filebeat](https://discuss.elastic.co/t/elasticsearch-permissions-required-by-filebeat/155185)
**Author:** [@alastairs](https://discuss.elastic.co/u/alastairs)\ **Replies:** 6\ **Last updated:** [November 2, 2018, 8:08pm UTC](https://discuss.elastic.co/t/elasticsearch-permissions-required-by-filebeat/155185 "2018-11-02T20:08:42Z")
I'm looking to secure my Elastic Cloud deployment by creating additional users (and roles where necessary) on the system. I'm not comfortable using the root elastic username and password for writing logs from Filebeat, a… --- ## [Filebeat input: intentional delay for specific source](https://discuss.elastic.co/t/filebeat-input-intentional-delay-for-specific-source/155124)
**Author:** [@justinw](https://discuss.elastic.co/u/justinw)\ **Replies:** 2\ **Last updated:** [November 2, 2018, 5:24pm UTC](https://discuss.elastic.co/t/filebeat-input-intentional-delay-for-specific-source/155124 "2018-11-02T17:24:54Z")
Hi, Is there a way to intentionally rate limit a certain file (say, one of the matches in the log input pattern /var/log/all-my-services/\*.log)? The use case is that one service on a shared machines logs so much that i… --- ## [Filebeat could not connect to logstash on SSL](https://discuss.elastic.co/t/filebeat-could-not-connect-to-logstash-on-ssl/152723)
**Author:** [@lcui\_dxc](https://discuss.elastic.co/u/lcui_dxc)\ **Replies:** 5\ **Last updated:** [November 2, 2018, 2:12pm UTC](https://discuss.elastic.co/t/filebeat-could-not-connect-to-logstash-on-ssl/152723 "2018-11-02T14:12:34Z")
Hello there, I set up the SSL on all our ELK nodes (2 Elastinodes, L&K on another node), all work fine. Without setting up SSL on Beat hosts, everything worked fine. Now we were requested to set up the SSL/TLS between… --- ## [How to set environment variables globally in metric beats](https://discuss.elastic.co/t/how-to-set-environment-variables-globally-in-metric-beats/154816)
**Author:** [@karthick\_tgi](https://discuss.elastic.co/u/karthick_tgi)\ **Replies:** 3\ **Last updated:** [November 2, 2018, 11:34am UTC](https://discuss.elastic.co/t/how-to-set-environment-variables-globally-in-metric-beats/154816 "2018-11-02T11:34:48Z")
How to set environment variables globally in metric beats. For example, In metric beat folder inside the mysql.yml file defaultlly its hard coded like hosts: \["root:root@tcp(localhost:3306)/"\] username and password a… --- ## [How to make my filebeat read from the beginning in log file](https://discuss.elastic.co/t/how-to-make-my-filebeat-read-from-the-beginning-in-log-file/154930)
**Author:** [@rajkumar.m](https://discuss.elastic.co/u/rajkumar.m)\ **Replies:** 3\ **Last updated:** [November 2, 2018, 10:23am UTC](https://discuss.elastic.co/t/how-to-make-my-filebeat-read-from-the-beginning-in-log-file/154930 "2018-11-02T10:23:19Z")
How to make my filebeat read from the beginning in log file everytime. i am using filebeat client node is sending CSV formatted log file to the logstash node and i have CSV filter in logstash which is creating fields a… --- ## [Finding total latency(round trip time or response time) per flow?](https://discuss.elastic.co/t/finding-total-latency-round-trip-time-or-response-time-per-flow/154971)
**Author:** [@sohaibomr](https://discuss.elastic.co/u/sohaibomr)\ **Replies:** 3\ **Last updated:** [November 2, 2018, 9:36am UTC](https://discuss.elastic.co/t/finding-total-latency-round-trip-time-or-response-time-per-flow/154971 "2018-11-02T09:36:15Z")
Hey all, I am using packet beat to build a network flows monitoring application, I need to find average latency rate for each flow. I wanted to ask if this is possible with current fields exported by packetbeat or If no… --- ## [Setting the Document ID](https://discuss.elastic.co/t/setting-the-document-id/154913)
**Author:** [@Garry](https://discuss.elastic.co/u/Garry)\ **Replies:** 6\ **Last updated:** [November 2, 2018, 8:17am UTC](https://discuss.elastic.co/t/setting-the-document-id/154913 "2018-11-02T08:17:04Z")
I am looking to set the document id while ingesting data though filebeats. How would I achieve this? --- ## [Lists in Kubernetes Hints Based Autodiscover Pod Annotations](https://discuss.elastic.co/t/lists-in-kubernetes-hints-based-autodiscover-pod-annotations/154817)
**Author:** [@bagratte](https://discuss.elastic.co/u/bagratte)\ **Replies:** 2\ **Last updated:** [November 2, 2018, 7:05am UTC](https://discuss.elastic.co/t/lists-in-kubernetes-hints-based-autodiscover-pod-annotations/154817 "2018-11-02T07:05:40Z")
I can't seem to be able to use lists in pod annotations. This doesn't work at all (i. e. no metrics are being collected at all by logstash module): annotations: co.elastic.metrics/module: logstash co.elastic.metrics… --- ## [Multi line pattern for different services to one logstash](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848)
**Author:** [@kuna](https://discuss.elastic.co/u/kuna)\ **Replies:** 7\ **Last updated:** [November 2, 2018, 5:39am UTC](https://discuss.elastic.co/t/multi-line-pattern-for-different-services-to-one-logstash/153848 "2018-11-02T05:39:18Z")
I have installed file beat agents for different service and pointed them to Logstash and applied my filter for it but noticed multiple logs are coming in a single log even though i was using multi line pattern to differe… --- ## [How to add custom routing ，version or id when index document](https://discuss.elastic.co/t/how-to-add-custom-routing-version-or-id-when-index-document/154951)
**Author:** [@luzhzhsoft](https://discuss.elastic.co/u/luzhzhsoft)\ **Replies:** 2\ **Last updated:** [November 2, 2018, 1:23am UTC](https://discuss.elastic.co/t/how-to-add-custom-routing-version-or-id-when-index-document/154951 "2018-11-02T01:23:28Z")
i write a s custom beat.And i want to add custom id,routing,version when index document.Because i use join mapping type,and routing must be same with parent routing rather than id [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=406) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=408)