# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=476 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 477 --- ## [Is it possible to add my own custom field using Winlogbeat?](https://discuss.elastic.co/t/is-it-possible-to-add-my-own-custom-field-using-winlogbeat/116789)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 6\ **Last updated:** [January 30, 2018, 8:38pm UTC](https://discuss.elastic.co/t/is-it-possible-to-add-my-own-custom-field-using-winlogbeat/116789 "2018-01-30T20:38:05Z")
Hi, I am trying to add my own custom field using winlogbeat. Can someone please say if that's p\[ossible? Thanks --- ## [Parsing typical logback log lines](https://discuss.elastic.co/t/parsing-typical-logback-log-lines/117665)
**Author:** [@gaukharaya](https://discuss.elastic.co/u/gaukharaya)\ **Replies:** 1\ **Last updated:** [January 30, 2018, 5:25pm UTC](https://discuss.elastic.co/t/parsing-typical-logback-log-lines/117665 "2018-01-30T17:25:59Z")
Hi, I am trying to send the following format log line from filebeat 6 to es6: %d{yyyy-MM-dd HH:mm:ss.SSS} \[%thread\] %-5level %logger{36} - %msg%n In my prospectors I specify: type: log Looking at fields.xml the def… --- ## [Where do get filebeat.template-es6x.json](https://discuss.elastic.co/t/where-do-get-filebeat-template-es6x-json/117598)
**Author:** [@gaukharaya](https://discuss.elastic.co/u/gaukharaya)\ **Replies:** 2\ **Last updated:** [January 30, 2018, 2:46pm UTC](https://discuss.elastic.co/t/where-do-get-filebeat-template-es6x-json/117598 "2018-01-30T14:46:55Z")
I am having a similar issue as posted here: But my filebeat is running on windows. The post suggests using filebeat.template-es6x.json. However I cannot find it my downloaded resource. I had to run a script to export… --- ## [Filebeat daemonset randomly deletes itself?!](https://discuss.elastic.co/t/filebeat-daemonset-randomly-deletes-itself/117600)
**Author:** [@Karl\_Stoney](https://discuss.elastic.co/u/Karl_Stoney)\ **Replies:** 1\ **Last updated:** [January 30, 2018, 2:46pm UTC](https://discuss.elastic.co/t/filebeat-daemonset-randomly-deletes-itself/117600 "2018-01-30T14:46:14Z")
Hey, I'm trying to run the filebeat docker container on my kubernetes cluster. It seems to come up... but then randomly, after about 30 seconds it disapears, and the daemonset, and associated configmaps are gone!? Log… --- ## [Single line with Filebeat to logstash](https://discuss.elastic.co/t/single-line-with-filebeat-to-logstash/117621)
**Author:** [@SarojELK](https://discuss.elastic.co/u/SarojELK)\ **Replies:** 1\ **Last updated:** [January 30, 2018, 2:45pm UTC](https://discuss.elastic.co/t/single-line-with-filebeat-to-logstash/117621 "2018-01-30T14:45:19Z")
Hi, is there any option with filebeat to extract lines only matching keywords from my input file ? normally my log file size is 25MB /day and similar to that i have almost more than 100+ files where i want to index only … --- ## [How to set mapping in filebeats?](https://discuss.elastic.co/t/how-to-set-mapping-in-filebeats/117578)
**Author:** [@Feng\_Mi](https://discuss.elastic.co/u/Feng_Mi)\ **Replies:** 1\ **Last updated:** [January 30, 2018, 8:54am UTC](https://discuss.elastic.co/t/how-to-set-mapping-in-filebeats/117578 "2018-01-30T08:54:17Z")
I want set the mapping in filebeat config. For example, there is a field name "use\_id", I want set the field type to text before create the index in es. But I don't know how to set the config. --- ## [ERR Kafka (topic=filebeat-test-logmiss30): dropping too large message of size 3463](https://discuss.elastic.co/t/err-kafka-topic-filebeat-test-logmiss30-dropping-too-large-message-of-size-3463/117003)
**Author:** [@wangyanzhao](https://discuss.elastic.co/u/wangyanzhao)\ **Replies:** 6\ **Last updated:** [January 30, 2018, 5:02am UTC](https://discuss.elastic.co/t/err-kafka-topic-filebeat-test-logmiss30-dropping-too-large-message-of-size-3463/117003 "2018-01-30T05:02:04Z")
I use filebeat to publish log to kafka, but get "2018/01/25 08:39:54.536086 client.go:203: ERR Kafka (topic=filebeat-test-logmiss30): dropping too large message of size 3463." configurations: Filebeat(6.1.2) output… --- ## [Winlogbeat processor not dropping events](https://discuss.elastic.co/t/winlogbeat-processor-not-dropping-events/115634)
**Author:** [@saikrishnagaddipati](https://discuss.elastic.co/u/saikrishnagaddipati)\ **Replies:** 7\ **Last updated:** [January 30, 2018, 3:49am UTC](https://discuss.elastic.co/t/winlogbeat-processor-not-dropping-events/115634 "2018-01-30T03:49:51Z")
I want to drop events if provider is not Service Control Manager or mfehidk or PRIVMAN AND if event\_id is not 7036 or 516 or 100. However this doesnot seems to work i am getting events that should be blocked? Any sugges… --- ## [Can we fill up some variable dynamically using jSON field?](https://discuss.elastic.co/t/can-we-fill-up-some-variable-dynamically-using-json-field/116765)
**Author:** [@jpereira](https://discuss.elastic.co/u/jpereira)\ **Replies:** 5\ **Last updated:** [January 29, 2018, 7:47pm UTC](https://discuss.elastic.co/t/can-we-fill-up-some-variable-dynamically-using-json-field/116765 "2018-01-29T19:47:39Z")
Hi there, In my system, we have several log types/levels that are based on the name "log\_type", e.g: "python\_log", "cache\_log", ... that are coming to the single log file "mylog.log" I am wondering to do something like… --- ## [Filebeat multiline docker logs](https://discuss.elastic.co/t/filebeat-multiline-docker-logs/117336)
**Author:** [@meril](https://discuss.elastic.co/u/meril)\ **Replies:** 5\ **Last updated:** [January 29, 2018, 3:55pm UTC](https://discuss.elastic.co/t/filebeat-multiline-docker-logs/117336 "2018-01-29T15:55:12Z")
Hi In production system we use filebeat 6.1 to manage docker log files with format like: ... {"log":"2017-11-26T16:59:56.912-0000 - INFO - blahblah","stream":"stdout","time":"2017-11-26T16:59:56.91609507Z"} ... Conf… --- ## [Parsing multiline logs](https://discuss.elastic.co/t/parsing-multiline-logs/117238)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 3\ **Last updated:** [January 29, 2018, 3:18pm UTC](https://discuss.elastic.co/t/parsing-multiline-logs/117238 "2018-01-29T15:18:51Z")
I am attempting to index a log file that consist of multilines and it appears one set of multilines is parsing correctly but the other isn't. Its being parsed as a separate log entry. Sample Log file: 2017-12-12 02:44:… --- ## [Filebeat to Logstash - "client is not connected"](https://discuss.elastic.co/t/filebeat-to-logstash-client-is-not-connected/116932)
**Author:** [@ijazadm](https://discuss.elastic.co/u/ijazadm)\ **Replies:** 31\ **Last updated:** [January 29, 2018, 1:29pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-client-is-not-connected/116932 "2018-01-29T13:29:24Z")
filebeat , logstash , elasticsearch : all latest versiions Hi , I have very strange problem , filebeat sends data to logstash , but after sometime it stops , errors in the log: Failed to publish events caused by: cli… --- ## [Server version: Apache/2.4.6 (CentOS) error fields in metricbeat](https://discuss.elastic.co/t/server-version-apache-2-4-6-centos-error-fields-in-metricbeat/117316)
**Author:** [@prophoto](https://discuss.elastic.co/u/prophoto)\ **Replies:** 1\ **Last updated:** [January 29, 2018, 12:03pm UTC](https://discuss.elastic.co/t/server-version-apache-2-4-6-centos-error-fields-in-metricbeat/117316 "2018-01-29T12:03:37Z")
2018-01-27T16:50:30Z ERR Error on field 'user': Missing field: user, Error: Key CPUUser not found 2018-01-27T16:50:30Z ERR Error on field 'system': Missing field: system, Error: Key CPUSystem not found 2018-01-27T16:… --- ## [Getting error while monitoring JVM via metricbeat](https://discuss.elastic.co/t/getting-error-while-monitoring-jvm-via-metricbeat/117403)
**Author:** [@Rahulkumar](https://discuss.elastic.co/u/Rahulkumar)\ **Replies:** 1\ **Last updated:** [January 29, 2018, 11:53am UTC](https://discuss.elastic.co/t/getting-error-while-monitoring-jvm-via-metricbeat/117403 "2018-01-29T11:53:32Z")
Hi Team, I am using simple commands to monitor JVM attributes with the help of metricbeat. - module: jolokia metricsets: \["jmx"\] enabled: true period: 1s hosts: \["host:8075"\] namespace: "metrics" jmx.mappin… --- ## [Conditional Filebeat output based on document type](https://discuss.elastic.co/t/conditional-filebeat-output-based-on-document-type/117233)
**Author:** [@tsullivan93](https://discuss.elastic.co/u/tsullivan93)\ **Replies:** 1\ **Last updated:** [January 29, 2018, 11:25am UTC](https://discuss.elastic.co/t/conditional-filebeat-output-based-on-document-type/117233 "2018-01-29T11:25:03Z")
Currently I have filebeat on a server with multiple different application logs, so I have multiple prospectors grabbing and tagging the different application logs. The logs have different formats so one of them needs to… --- ## [Filebeat config is OK, but Not Started & Not Working](https://discuss.elastic.co/t/filebeat-config-is-ok-but-not-started-not-working/117301)
**Author:** [@omid](https://discuss.elastic.co/u/omid)\ **Replies:** 7\ **Last updated:** [January 29, 2018, 11:19am UTC](https://discuss.elastic.co/t/filebeat-config-is-ok-but-not-started-not-working/117301 "2018-01-29T11:19:31Z")
im installed a filebeat on server Which is to send nginx log to logstash im download filebeat .deb package (x64) and install "dpkg --install filebeat ... .deb" and successfully install . But service filebeat not work… --- ## [Filebeat service Start but not work, Error](https://discuss.elastic.co/t/filebeat-service-start-but-not-work-error/117435)
**Author:** [@omid](https://discuss.elastic.co/u/omid)\ **Replies:** 1\ **Last updated:** [January 29, 2018, 11:11am UTC](https://discuss.elastic.co/t/filebeat-service-start-but-not-work-error/117435 "2018-01-29T11:11:29Z")
2018-01-29T13:59:37+03:30 INFO Non-zero metrics in the last 30s: beat.info.uptime.ms=30001 beat.memstats.gc\_next=10555920 beat.memstats.memory\_alloc=5393592 beat.memstats.memory\_total=19446152 filebeat.harvester.open\_fi… --- ## [Packetbeat on Win2016 Core](https://discuss.elastic.co/t/packetbeat-on-win2016-core/117370)
**Author:** [@antwan](https://discuss.elastic.co/u/antwan)\ **Replies:** 1\ **Last updated:** [January 28, 2018, 2:29pm UTC](https://discuss.elastic.co/t/packetbeat-on-win2016-core/117370 "2018-01-28T14:29:53Z")
Hey, I am trying to deploy Packetbeat 6.1.2 on a Windows 2016 Standard Core using Ansible. I install WinPcap using the win\_chocolatey module in Ansible, and then Packetbeat calling the supplied powershell-script. I use … --- ## [Understanding HA with Kafka](https://discuss.elastic.co/t/understanding-ha-with-kafka/116162)
**Author:** [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\ **Replies:** 22\ **Last updated:** [January 27, 2018, 4:18pm UTC](https://discuss.elastic.co/t/understanding-ha-with-kafka/116162 "2018-01-27T16:18:33Z")
Hi, running 6.1.1. I have 3 Kafka brokers (Confluent 3.3.1e, Kafka 0.11.0). I physically downed 2 of the brokers for 10 minutes, but logs kept getting indexed! The topic has replica count 1. Unless I don't understand … --- ## [Log with json value and timestamp value from syslog](https://discuss.elastic.co/t/log-with-json-value-and-timestamp-value-from-syslog/117245)
**Author:** [@ramzey1981](https://discuss.elastic.co/u/ramzey1981)\ **Replies:** 3\ **Last updated:** [January 26, 2018, 8:06pm UTC](https://discuss.elastic.co/t/log-with-json-value-and-timestamp-value-from-syslog/117245 "2018-01-26T20:06:45Z")
we have a python application logging to syslog in json format but syslog allows appends a timestamp with the json message, Jan 26 11:07:14 mkobit-ThinkPad {"@source\_host": "mkobit-ThinkPad", "module": "pyscratch", "line… --- ## [Heartbeat Dashboard Questions](https://discuss.elastic.co/t/heartbeat-dashboard-questions/117206)
**Author:** [@Ninoslav](https://discuss.elastic.co/u/Ninoslav)\ **Replies:** 7\ **Last updated:** [January 26, 2018, 3:55pm UTC](https://discuss.elastic.co/t/heartbeat-dashboard-questions/117206 "2018-01-26T15:55:05Z")
Hi guys, I have a few questions about hearthbeat. I am using ELK stack 6.0.0, and I have successfully installed hearthbeat, hearthbeat template, and dashboard. But I\`am experiencing some limitations, which I'm not sure… --- ## [Winlogbeat -\> Logstash -\> ES failing to write](https://discuss.elastic.co/t/winlogbeat-logstash-es-failing-to-write/116874)
**Author:** [@cymlog](https://discuss.elastic.co/u/cymlog)\ **Replies:** 5\ **Last updated:** [January 26, 2018, 3:41pm UTC](https://discuss.elastic.co/t/winlogbeat-logstash-es-failing-to-write/116874 "2018-01-26T15:41:37Z")
I am am experimenting with Winlogbeat, Logstash, ES. I am experiencing intermittent log writes into ES from logstash I have configured a WinServer 2008 R2 Vm with Winlogbeat used the following config to logstash (ensur… --- ## [Persist measures on the monitored host in case of network faillure](https://discuss.elastic.co/t/persist-measures-on-the-monitored-host-in-case-of-network-faillure/117212)
**Author:** [@anon24174512](https://discuss.elastic.co/u/anon24174512)\ **Replies:** 1\ **Last updated:** [January 26, 2018, 3:34pm UTC](https://discuss.elastic.co/t/persist-measures-on-the-monitored-host-in-case-of-network-faillure/117212 "2018-01-26T15:34:26Z")
Hello, Is there a way from the metricbeat sink to persist the measures on the monitored node in case of network failure so as to send them back to elasticsearch once the network is back and not loosing them ? Or do I h… --- ## [MetricBeat - Perfmon Counters](https://discuss.elastic.co/t/metricbeat-perfmon-counters/116150)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 7\ **Last updated:** [January 26, 2018, 3:23pm UTC](https://discuss.elastic.co/t/metricbeat-perfmon-counters/116150 "2018-01-26T15:23:01Z")
Been playing with the ELK stack pretty heavily the last week or so running on a Windows platform. I'd like to collect per CPU metrics and have them display on a time series. This is what I have in the windows.yml modul… --- ## [Filebeat not able to connect to Logstatsh - Getting ERR Failed to connect: dial tcp XX.XX.XXX.X:5044: i/o timeout](https://discuss.elastic.co/t/filebeat-not-able-to-connect-to-logstatsh-getting-err-failed-to-connect-dial-tcp-xx-xx-xxx-x-5044-i-o-timeout/117202)
**Author:** [@Vijai](https://discuss.elastic.co/u/Vijai)\ **Replies:** 1\ **Last updated:** [January 26, 2018, 2:41pm UTC](https://discuss.elastic.co/t/filebeat-not-able-to-connect-to-logstatsh-getting-err-failed-to-connect-dial-tcp-xx-xx-xxx-x-5044-i-o-timeout/117202 "2018-01-26T14:41:59Z")
Hello, I'm trying to do a POC on ELK + Filebeat. I configured the entire setup on my local machine and it works fine. Filebeats is able to establish a connection to Logstash. But when I installed Filebeats on a remote s… --- ## [Filebeat multiline works only in pair with logstash](https://discuss.elastic.co/t/filebeat-multiline-works-only-in-pair-with-logstash/117191)
**Author:** [@samdocker](https://discuss.elastic.co/u/samdocker)\ **Replies:** 2\ **Last updated:** [January 26, 2018, 2:18pm UTC](https://discuss.elastic.co/t/filebeat-multiline-works-only-in-pair-with-logstash/117191 "2018-01-26T14:18:18Z")
Hello. Weird topic, but this is how see this problem. So. Currently we are parsing multiline on logstash side. I want to move multiline to filebeat. Now. I comment out codec =\> multiline { ... } restart logstash … --- ## [MySQL (MariaDB) Overview dashboard empty](https://discuss.elastic.co/t/mysql-mariadb-overview-dashboard-empty/117115)
**Author:** [@castle9mm](https://discuss.elastic.co/u/castle9mm)\ **Replies:** 1\ **Last updated:** [January 26, 2018, 11:07am UTC](https://discuss.elastic.co/t/mysql-mariadb-overview-dashboard-empty/117115 "2018-01-26T11:07:44Z")
Hello, sorry for what's probably a basic beginner issue but my MySQL Overview dashboard in Kibana is empty. I'm trying to use Filebeat with AutoDiscover to capture the mysql error & slow query logs from a MariaDB contai… --- ## [Filebeat write:connection reset by peer](https://discuss.elastic.co/t/filebeat-write-connection-reset-by-peer/114890)
**Author:** [@DavidDebray](https://discuss.elastic.co/u/DavidDebray)\ **Replies:** 8\ **Last updated:** [January 24, 2018, 1:50pm UTC](https://discuss.elastic.co/t/filebeat-write-connection-reset-by-peer/114890 "2018-01-24T13:50:40Z")
Hi everyone! Here's my problem, when I start Filebeat on my clients I keep having the same error: ERR Failed to publish events: write tcp client\_ip:41352-\>elk\_ip:5044: write: connection reset by peer Logstash, Elastic… --- ## [Filebeat drop events with multiple words](https://discuss.elastic.co/t/filebeat-drop-events-with-multiple-words/116768)
**Author:** [@thinkdffrnt](https://discuss.elastic.co/u/thinkdffrnt)\ **Replies:** 5\ **Last updated:** [January 26, 2018, 4:57am UTC](https://discuss.elastic.co/t/filebeat-drop-events-with-multiple-words/116768 "2018-01-26T04:57:36Z")
I'm trying to drop events with multiple words like Tuple, TUPLE, tuple Complete config doesn't seem to work with one word filter too filebeat: registry\_file: /var/run/filebeat/.gl-filebeat-registry spool\_size: 2… --- ## [Convert nanocore to percentage](https://discuss.elastic.co/t/convert-nanocore-to-percentage/116538)
**Author:** [@kanthimathi](https://discuss.elastic.co/u/kanthimathi)\ **Replies:** 5\ **Last updated:** [January 25, 2018, 9:34pm UTC](https://discuss.elastic.co/t/convert-nanocore-to-percentage/116538 "2018-01-25T21:34:39Z")
Is there any way to convert "kubernetes.container.memory.usage" from nanocore to percentage? or any term little bigger than nanocore? \[51%20AM\] Need to convert this from nanocore to percentage? Thanks, Kanthi. [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=475) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=477)