# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=328 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 329 --- ## [ERROR logstash/async.go:256 Failed to publish events caused by: write tcp 10.XXX.XX.XX:43522-\>10.XXX.XX.XX:5044: i/o timeout 2019-08-06T16:04:30.967+0800 ERROR pipeline/output.go:121 Failed to publish events:IOtimeout](https://discuss.elastic.co/t/error-logstash-async-go-256-failed-to-publish-events-caused-by-write-tcp-10-xxx-xx-xx-43522-10-xxx-xx-xx-i-o-timeout-2019-08-06t1630-967-0800-error-pipeline-output-go-121-failed-to-publish-events-iotimeout/194355)
**Author:** [@uday\_kumar1](https://discuss.elastic.co/u/uday_kumar1)\ **Replies:** 2\ **Last updated:** [August 12, 2019, 6:06am UTC](https://discuss.elastic.co/t/error-logstash-async-go-256-failed-to-publish-events-caused-by-write-tcp-10-xxx-xx-xx-43522-10-xxx-xx-xx-i-o-timeout-2019-08-06t1630-967-0800-error-pipeline-output-go-121-failed-to-publish-events-iotimeout/194355 "2019-08-12T06:06:20Z")
Hello There, Currently we are testing the filebeat service to send the logs from client to server. In this process I am getting below error. 2019-08-06T16:04:29.146+0800 ERROR logstash/async.go:256 Failed to pub… --- ## [Beat package issue for windows](https://discuss.elastic.co/t/beat-package-issue-for-windows/194165)
**Author:** [@balamurugan\_ravi](https://discuss.elastic.co/u/balamurugan_ravi)\ **Replies:** 4\ **Last updated:** [August 8, 2019, 4:12am UTC](https://discuss.elastic.co/t/beat-package-issue-for-windows/194165 "2019-08-08T04:12:26Z")
Hi , We are getting below issue , when we run "make release" " 05:31:30 Found Elastic Beats dir at /root/go/src/github.com/scb/appbeat/vendor/github.com/elastic/beats Generated fields.yml for appbeat to /root/go/src/g… --- ## [Beat processor drop fields](https://discuss.elastic.co/t/beat-processor-drop-fields/192213)
**Author:** [@ajhstn](https://discuss.elastic.co/u/ajhstn)\ **Replies:** 3\ **Last updated:** [August 12, 2019, 3:16am UTC](https://discuss.elastic.co/t/beat-processor-drop-fields/192213 "2019-08-12T03:16:26Z")
Hello - what would the best way to "drop all message fields, except for x or y o z"? I want to start by dropping the message field for all events, but then only include it on a few using a when.or condition? --- ## [Nginx module is trying to parsing each log entry as access and error log](https://discuss.elastic.co/t/nginx-module-is-trying-to-parsing-each-log-entry-as-access-and-error-log/193631)
**Author:** [@Mawalu](https://discuss.elastic.co/u/Mawalu)\ **Replies:** 8\ **Last updated:** [August 12, 2019, 3:13am UTC](https://discuss.elastic.co/t/nginx-module-is-trying-to-parsing-each-log-entry-as-access-and-error-log/193631 "2019-08-12T03:13:20Z")
I've setup filebeat in docker with hints to collect logs of my nginx container. In ES all log entries appear twice, once parsed by the correct fileset (access or error) and once unparsed with a grok error by the other fi… --- ## [Nginx logs from stdout coming to error pipeline instead of access pipeline](https://discuss.elastic.co/t/nginx-logs-from-stdout-coming-to-error-pipeline-instead-of-access-pipeline/194772)
**Author:** [@poma](https://discuss.elastic.co/u/poma)\ **Replies:** 1\ **Last updated:** [August 12, 2019, 2:18am UTC](https://discuss.elastic.co/t/nginx-logs-from-stdout-coming-to-error-pipeline-instead-of-access-pipeline/194772 "2019-08-12T02:18:13Z")
edit: read below for update to initial question I'm getting Provided Grok expressions do not match field value even though \_simulate works with exact same string. Error message: "message": "172.18.0.2 - - \[12/Aug/2019… --- ## [Filebeat - Kubernetes - logs in plaintest instead JSON](https://discuss.elastic.co/t/filebeat-kubernetes-logs-in-plaintest-instead-json/194608)
**Author:** [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Replies:** 3\ **Last updated:** [August 10, 2019, 11:51am UTC](https://discuss.elastic.co/t/filebeat-kubernetes-logs-in-plaintest-instead-json/194608 "2019-08-10T11:51:01Z")
Hi, I am using 7.1.1 version elastic stack setup. Filebeat running on kubernetes cluster as pod and sending logs to ELK. I could see all pod logs are in JSON when opening in vi editor. example sudo vi /datavg/docke… --- ## [Add\_fields processor not working](https://discuss.elastic.co/t/add-fields-processor-not-working/194688)
**Author:** [@ajhstn](https://discuss.elastic.co/u/ajhstn)\ **Replies:** 1\ **Last updated:** [August 10, 2019, 4:33am UTC](https://discuss.elastic.co/t/add-fields-processor-not-working/194688 "2019-08-10T04:33:13Z")
Hello - can you spot any problem with this processor. when i .\\winlogbeat.exe test config i get a good Config OK as expected, however when events that i feel should match the processor occur, the field is not being adde… --- ## [Elastic Cloud Keystore](https://discuss.elastic.co/t/elastic-cloud-keystore/192187)
**Author:** [@ajhstn](https://discuss.elastic.co/u/ajhstn)\ **Replies:** 5\ **Last updated:** [August 10, 2019, 4:13am UTC](https://discuss.elastic.co/t/elastic-cloud-keystore/192187 "2019-08-10T04:13:38Z")
In Elastic cloud i have configured a Keystore setting named ES\_PWD, its a single string containing a value of elastic:myelasticpassword. I tried to use this in my beat configuration, cloud.auth: ${ES\_PWD} but when i tes… --- ## [Certificate monitoring using Heartbeat](https://discuss.elastic.co/t/certificate-monitoring-using-heartbeat/191657)
**Author:** [@Freddy.Raj](https://discuss.elastic.co/u/Freddy.Raj)\ **Replies:** 1\ **Last updated:** [August 10, 2019, 2:19am UTC](https://discuss.elastic.co/t/certificate-monitoring-using-heartbeat/191657 "2019-08-10T02:19:20Z")
Hi Team, I am currently setting up heartbeat to monitor URLs of our application. However would like to know if we can monitor https and captyre cert expiry details as well in the kibana dashboard. Please help. Runnin… --- ## [Not able to create dashboard for heartbeat 7.2 in kibana through setup command](https://discuss.elastic.co/t/not-able-to-create-dashboard-for-heartbeat-7-2-in-kibana-through-setup-command/191978)
**Author:** [@Raghav\_Vasishth](https://discuss.elastic.co/u/Raghav_Vasishth)\ **Replies:** 1\ **Last updated:** [August 10, 2019, 2:16am UTC](https://discuss.elastic.co/t/not-able-to-create-dashboard-for-heartbeat-7-2-in-kibana-through-setup-command/191978 "2019-08-10T02:16:30Z")
Not able to create dashboard in kibana through setup command . Is there any other way to create dashboard ? In heartbeat 6.3 below command was working and giving the below mentioned output Command: sudo heartbeat setu… --- ## [Send heartbeat logs to multiple topics in Kafka](https://discuss.elastic.co/t/send-heartbeat-logs-to-multiple-topics-in-kafka/192306)
**Author:** [@Roopam\_Rajvanshi](https://discuss.elastic.co/u/Roopam_Rajvanshi)\ **Replies:** 1\ **Last updated:** [August 10, 2019, 2:15am UTC](https://discuss.elastic.co/t/send-heartbeat-logs-to-multiple-topics-in-kafka/192306 "2019-08-10T02:15:04Z")
Is it possible to send heartbeat logs to multiple topics in kafka based on a condition if a field exists or not? I want to send the logs to multiple topics in case the machine being pinged is down. The reason being that … --- ## [Injest multiple json files into elasticsearch using filebeat as a different index for each file](https://discuss.elastic.co/t/injest-multiple-json-files-into-elasticsearch-using-filebeat-as-a-different-index-for-each-file/194652)
**Author:** [@sadhika7](https://discuss.elastic.co/u/sadhika7)\ **Replies:** 4\ **Last updated:** [August 9, 2019, 6:32pm UTC](https://discuss.elastic.co/t/injest-multiple-json-files-into-elasticsearch-using-filebeat-as-a-different-index-for-each-file/194652 "2019-08-09T18:32:40Z")
I am trying to injest multiple json files into elasticsearch using filebeat but each json file as a different index name in elasticsearch. Do I also need to use logstash for this? I am able to install a single json file… --- ## [Filebeat config prospectors paths with pattern](https://discuss.elastic.co/t/filebeat-config-prospectors-paths-with-pattern/194645)
**Author:** [@jabravoa](https://discuss.elastic.co/u/jabravoa)\ **Replies:** 0\ **Last updated:** [August 9, 2019, 3:54pm UTC](https://discuss.elastic.co/t/filebeat-config-prospectors-paths-with-pattern/194645 "2019-08-09T15:54:32Z")
Hello. I want to configure my filebeat prospectors paths with a pattern. My directory that contains the log file is like /scripts/09082019/file.log and this one generates every day changing just the date pattern ddMMYY… --- ## [Filebeat doesn't automatically send new files](https://discuss.elastic.co/t/filebeat-doesnt-automatically-send-new-files/194390)
**Author:** [@luc1](https://discuss.elastic.co/u/luc1)\ **Replies:** 2\ **Last updated:** [August 9, 2019, 3:35pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-automatically-send-new-files/194390 "2019-08-09T15:35:42Z")
Hi, I configured filebeat so that it ships new files coming from a sftp protocol to a repertory "Data", to logstash, and it doesn't. If I modify the file myself (like adding " " at the beginning of the file) once it is … --- ## [Nginx logs in Kibana log streaming!](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944)
**Author:** [@sv\_bcvt92](https://discuss.elastic.co/u/sv_bcvt92)\ **Replies:** 4\ **Last updated:** [August 9, 2019, 3:32pm UTC](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944 "2019-08-09T15:32:12Z")
Hi Guru's, I'm setting up Laravel with Nginx. And I expose Laravel access logs to Nginx logs as config below: server { listen 80; listen \[::\]:80 ipv6only=on; server\_name abcyxyz.ap-northeast-1.… --- ## [Filebeat unable to ship non modified files to logstash](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955)
**Author:** [@saisimo02](https://discuss.elastic.co/u/saisimo02)\ **Replies:** 4\ **Last updated:** [August 9, 2019, 3:30pm UTC](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955 "2019-08-09T15:30:17Z")
Hello, I configured filebeat.yml to send data to logstash, I just specified where my data are stored and the ip address of my logstash. I have a python script to generate data, if I don't change any line in my data, fi… --- ## [Filebeat \[63.0-Windows\] tail\_files not working](https://discuss.elastic.co/t/filebeat-63-0-windows-tail-files-not-working/193816)
**Author:** [@111191](https://discuss.elastic.co/u/111191)\ **Replies:** 3\ **Last updated:** [August 9, 2019, 3:27pm UTC](https://discuss.elastic.co/t/filebeat-63-0-windows-tail-files-not-working/193816 "2019-08-09T15:27:52Z")
Hi, I enabled tail\_files in my filebeat.yml but filebeat yet send all the file instead of only the new lines. I stopped the service and I deleted the registry file from %programdata%\\filebeat (I'm using windows) My y… --- ## [Beats (6.8.2) via Logstash using collectd index when more than one host specified](https://discuss.elastic.co/t/beats-6-8-2-via-logstash-using-collectd-index-when-more-than-one-host-specified/194510)
**Author:** [@TomD](https://discuss.elastic.co/u/TomD)\ **Replies:** 2\ **Last updated:** [August 9, 2019, 3:26pm UTC](https://discuss.elastic.co/t/beats-6-8-2-via-logstash-using-collectd-index-when-more-than-one-host-specified/194510 "2019-08-09T15:26:06Z")
Hi If I modify the default config (/etc/filebeat/filebeat.yml) to disable output.elasticsearch and add output.logstash like below I get duplicate updates. One to filebeat-6.8.2-2019.08.08 and another to collectd-2019.08… --- ## [Filebeat Multiline for logs starting in the same way](https://discuss.elastic.co/t/filebeat-multiline-for-logs-starting-in-the-same-way/194587)
**Author:** [@lisa94](https://discuss.elastic.co/u/lisa94)\ **Replies:** 1\ **Last updated:** [August 9, 2019, 3:24pm UTC](https://discuss.elastic.co/t/filebeat-multiline-for-logs-starting-in-the-same-way/194587 "2019-08-09T15:24:09Z")
Hello everybody, I'm new to the community and I wanted to ask you a question. I have a log file that is sent to Logstash through Filebeat. This log file is composed of lines like this: 2019.08.09 11:19:22 INFO \[sock… --- ## [Repeat testing/simulation with fixed log scenarios](https://discuss.elastic.co/t/repeat-testing-simulation-with-fixed-log-scenarios/194386)
**Author:** [@Kevin\_Taylor](https://discuss.elastic.co/u/Kevin_Taylor)\ **Replies:** 5\ **Last updated:** [August 9, 2019, 3:10pm UTC](https://discuss.elastic.co/t/repeat-testing-simulation-with-fixed-log-scenarios/194386 "2019-08-09T15:10:11Z")
Hi All, I'm fairly new to the elastic stack but I've successfully created various sandbox environments (using docker compose) chaining FileBeat, WinLogBeat and PacketBeat to LogStash for example. I've worked a lot with … --- ## [Filebeat 7.3.0 Autodiscover configuration introduction yields "Cannot connect to the Docker daemon" error](https://discuss.elastic.co/t/filebeat-7-3-0-autodiscover-configuration-introduction-yields-cannot-connect-to-the-docker-daemon-error/193974)
**Author:** [@nmccoy](https://discuss.elastic.co/u/nmccoy)\ **Replies:** 3\ **Last updated:** [August 9, 2019, 1:19pm UTC](https://discuss.elastic.co/t/filebeat-7-3-0-autodiscover-configuration-introduction-yields-cannot-connect-to-the-docker-daemon-error/193974 "2019-08-09T13:19:14Z")
I would like to roll out an ELK stack using filebeat to transport logs from custom docker containers. The OS is Ubuntu 18.04. I am trying to do this by using the filebeat autodiscover feature by following the documentat… --- ## [Is Beats central management requires Restart of agent when we have update in central management configuration](https://discuss.elastic.co/t/is-beats-central-management-requires-restart-of-agent-when-we-have-update-in-central-management-configuration/194439)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 4\ **Last updated:** [August 9, 2019, 1:04pm UTC](https://discuss.elastic.co/t/is-beats-central-management-requires-restart-of-agent-when-we-have-update-in-central-management-configuration/194439 "2019-08-09T13:04:26Z")
Hi Team, I am trying to setup beats central management, at the beginning of enrollment I have configured /tmp/test1.txt file for monitoring, then started beats agent, I am able to see the events in logstash. After that… --- ## [Separate index per application or name space](https://discuss.elastic.co/t/separate-index-per-application-or-name-space/194257)
**Author:** [@raju.d](https://discuss.elastic.co/u/raju.d)\ **Replies:** 6\ **Last updated:** [August 9, 2019, 12:06pm UTC](https://discuss.elastic.co/t/separate-index-per-application-or-name-space/194257 "2019-08-09T12:06:43Z")
Hello, I have setup filebeat in kubernetes clusters, that is collecting logs from all containers with a specific annotation. The annotation its looking for is kubernetes.labels.apps Elastic search cluster is running in… --- ## [MS SQL Transaction Log Errors](https://discuss.elastic.co/t/ms-sql-transaction-log-errors/194220)
**Author:** [@SvenC56](https://discuss.elastic.co/u/SvenC56)\ **Replies:** 5\ **Last updated:** [August 9, 2019, 11:50am UTC](https://discuss.elastic.co/t/ms-sql-transaction-log-errors/194220 "2019-08-09T11:50:01Z")
Hi, I'm running the MSSQL Beat and I get some error messages. This is my connection string: sqlserver://login:password@localhost:1433/instance\_name In the explorer field I get the following error message in the trans… --- ## [Auditbeat shards failing](https://discuss.elastic.co/t/auditbeat-shards-failing/194428)
**Author:** [@nilubkal](https://discuss.elastic.co/u/nilubkal)\ **Replies:** 2\ **Last updated:** [August 9, 2019, 11:24am UTC](https://discuss.elastic.co/t/auditbeat-shards-failing/194428 "2019-08-09T11:24:53Z")
Hello, since i deployed an ELK 7.2 stack ( 4 days ago ) the auditbeat shards are failing with 3 out of 4 shards when i try to use the defaulf \[\[Auditbeat Auditd\] Overview ECS\] dashboard ( or any of the other auditbeat… --- ## [Kubernetes Metric beat http configuration issues](https://discuss.elastic.co/t/kubernetes-metric-beat-http-configuration-issues/194593)
**Author:** [@Vijaybhaskar\_Vishnub](https://discuss.elastic.co/u/Vijaybhaskar_Vishnub)\ **Replies:** 5\ **Last updated:** [August 9, 2019, 10:49am UTC](https://discuss.elastic.co/t/kubernetes-metric-beat-http-configuration-issues/194593 "2019-08-09T10:49:15Z")
Hi In Kubernetes i couldn't enable http module. Following is metric beat configuration file: Its just ignoring http module. Where as stand alone setup locally working fine after enabling http module #Starting apiVer… --- ## [Monitor Tomcat services using Beats](https://discuss.elastic.co/t/monitor-tomcat-services-using-beats/194222)
**Author:** [@Tinkerbell](https://discuss.elastic.co/u/Tinkerbell)\ **Replies:** 9\ **Last updated:** [August 9, 2019, 9:11am UTC](https://discuss.elastic.co/t/monitor-tomcat-services-using-beats/194222 "2019-08-09T09:11:18Z")
I am using Beats-\>ElasticSearch-\>Kibana for viewing the metrics. Now I need to monitor the webapplications that are deployed on a tomcat server. How can we achieve this? Please help. --- ## [Send metadata from filebeat to logstash](https://discuss.elastic.co/t/send-metadata-from-filebeat-to-logstash/194030)
**Author:** [@bm\_rec](https://discuss.elastic.co/u/bm_rec)\ **Replies:** 4\ **Last updated:** [August 9, 2019, 3:57am UTC](https://discuss.elastic.co/t/send-metadata-from-filebeat-to-logstash/194030 "2019-08-09T03:57:23Z")
Hi! I'd like to send a hostname and some other host metadata from filebeat to Logstash. Then Logstash saves files with a name like: Summaryfile { path =\> "~/%{+YYYY-MM-dd}/%{component}/%{\[@metadata\]\[host… --- ## [Filebeats to Elasticsearch directly?](https://discuss.elastic.co/t/filebeats-to-elasticsearch-directly/194477)
**Author:** [@cybercorp](https://discuss.elastic.co/u/cybercorp)\ **Replies:** 1\ **Last updated:** [August 8, 2019, 7:59pm UTC](https://discuss.elastic.co/t/filebeats-to-elasticsearch-directly/194477 "2019-08-08T19:59:56Z")
Hello Filebeats seems to have many modules available like system, apache, and others. Since modules are responsible for parsing the data, if I send data directly from Filebeats to ES, will those logs supported by module… --- ## [How to do password variable substitution in beats install](https://discuss.elastic.co/t/how-to-do-password-variable-substitution-in-beats-install/194482)
**Author:** [@niudaye123](https://discuss.elastic.co/u/niudaye123)\ **Replies:** 1\ **Last updated:** [August 8, 2019, 7:50pm UTC](https://discuss.elastic.co/t/how-to-do-password-variable-substitution-in-beats-install/194482 "2019-08-08T19:50:39Z")
Hi, I would like to do password variable substitution in the beats configuration, something like this: ${beats\_password}, should I set the password value in elasticsearch keystore for that? How the value get passed to b… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=327) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=329)