# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=429 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 430 --- ## [Logstash not showing any data when input is configured with filebeat](https://discuss.elastic.co/t/logstash-not-showing-any-data-when-input-is-configured-with-filebeat/143566)
**Author:** [@Vijay\_Chander](https://discuss.elastic.co/u/Vijay_Chander)\ **Replies:** 3\ **Last updated:** [August 9, 2018, 12:24pm UTC](https://discuss.elastic.co/t/logstash-not-showing-any-data-when-input-is-configured-with-filebeat/143566 "2018-08-09T12:24:59Z")
filebeat.yml file filebeat.prospectors: type: log paths: "/home/ec2-user/logs/sas\_logs/\*.log" output.logstash: hosts: \["localhost:5044"\] logstash.conf file input { beats { port =\> "5044" } } filter { … --- ## [How to prevent adding extra fields by filebeat to the actual log records](https://discuss.elastic.co/t/how-to-prevent-adding-extra-fields-by-filebeat-to-the-actual-log-records/143095)
**Author:** [@ramanna\_hk](https://discuss.elastic.co/u/ramanna_hk)\ **Replies:** 3\ **Last updated:** [August 9, 2018, 5:42am UTC](https://discuss.elastic.co/t/how-to-prevent-adding-extra-fields-by-filebeat-to-the-actual-log-records/143095 "2018-08-09T05:42:15Z")
Hi, I am shipping log data to Kafka topic using file beat, here the problem is i don't want to have extra fields added by file-beat ex: file-beat version. is it possible? Thanks --- ## [Filebeat is running, but can't detect filelogs is changed](https://discuss.elastic.co/t/filebeat-is-running-but-cant-detect-filelogs-is-changed/143253)
**Author:** [@Quang\_Tho](https://discuss.elastic.co/u/Quang_Tho)\ **Replies:** 2\ **Last updated:** [August 9, 2018, 4:24am UTC](https://discuss.elastic.co/t/filebeat-is-running-but-cant-detect-filelogs-is-changed/143253 "2018-08-09T04:24:21Z")
I am deploy filebeat for shipping docker logs to elk. Filebeat is running but cannot ship log to logstash. is any missing here, please check for me. --- docker-compose.yml ----- version: "3.6" services: #############… --- ## [Filebeat skips fields for the first line in UTF8 with BOM encoding](https://discuss.elastic.co/t/filebeat-skips-fields-for-the-first-line-in-utf8-with-bom-encoding/143320)
**Author:** [@alatas](https://discuss.elastic.co/u/alatas)\ **Replies:** 2\ **Last updated:** [August 8, 2018, 8:11pm UTC](https://discuss.elastic.co/t/filebeat-skips-fields-for-the-first-line-in-utf8-with-bom-encoding/143320 "2018-08-08T20:11:06Z")
Hi, I'm using filebeat for providing logs to ELK stack. Log files are composed of json lines. I also added custom fields to output with filebeat. When I saved log files utf8 with bom signature, Filebeat doesn't add cust… --- ## [IIS Log message field doesn't always get parsed](https://discuss.elastic.co/t/iis-log-message-field-doesnt-always-get-parsed/143383)
**Author:** [@AlexB](https://discuss.elastic.co/u/AlexB)\ **Replies:** 11\ **Last updated:** [August 8, 2018, 6:57pm UTC](https://discuss.elastic.co/t/iis-log-message-field-doesnt-always-get-parsed/143383 "2018-08-08T18:57:56Z")
I'm pushing a few IIS logs to logstash and some of the "message" fields do not get parsed. I've tested the log lines in the "message" field against the pattern that's in the default.json injest and the lines were parsed … --- ## [Spooling Queue](https://discuss.elastic.co/t/spooling-queue/142797)
**Author:** [@BrandonF](https://discuss.elastic.co/u/BrandonF)\ **Replies:** 7\ **Last updated:** [August 8, 2018, 6:05pm UTC](https://discuss.elastic.co/t/spooling-queue/142797 "2018-08-08T18:05:04Z")
Attempting to play with the new spooling queue but we're having issues with the config around it. The most prevalent error when testing the config: spool.dat can not be locked right now Current config section: spool: … --- ## [Packetbeat for search query monitoring](https://discuss.elastic.co/t/packetbeat-for-search-query-monitoring/143406)
**Author:** [@Derek\_Liu](https://discuss.elastic.co/u/Derek_Liu)\ **Replies:** 3\ **Last updated:** [August 8, 2018, 5:57pm UTC](https://discuss.elastic.co/t/packetbeat-for-search-query-monitoring/143406 "2018-08-08T17:57:33Z")
I am trying to use packetbeat to monitor users' search queries and store them into elasticsearch using logstash. However, when I am trying to test it out on my local, packetbeat does not capture any of search queries I e… --- ## [Nginx logs on Kubernetes not parsed](https://discuss.elastic.co/t/nginx-logs-on-kubernetes-not-parsed/143446)
**Author:** [@iremmats](https://discuss.elastic.co/u/iremmats)\ **Replies:** 3\ **Last updated:** [August 8, 2018, 3:34pm UTC](https://discuss.elastic.co/t/nginx-logs-on-kubernetes-not-parsed/143446 "2018-08-08T15:34:57Z")
We are running Filebeat on Kubernetes to ingest nginx logs. Its going fine except for the fact that lines are not being parsed. The entire nginx log line shows up under "message" in elastic. "message": "10.199.12.66 - -… --- ## [Mixed document types: json and plain text](https://discuss.elastic.co/t/mixed-document-types-json-and-plain-text/143536)
**Author:** [@mimmus](https://discuss.elastic.co/u/mimmus)\ **Replies:** 1\ **Last updated:** [August 8, 2018, 2:36pm UTC](https://discuss.elastic.co/t/mixed-document-types-json-and-plain-text/143536 "2018-08-08T14:36:05Z")
We are migrating all of our applications to use the JSON encoded format, but until then we have some log files in json and some with normal logs. We cannot differentiate them using different paths because all of them ar… --- ## [Metricbeat data not flowing into elasticsearch](https://discuss.elastic.co/t/metricbeat-data-not-flowing-into-elasticsearch/143328)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 14\ **Last updated:** [August 8, 2018, 1:13pm UTC](https://discuss.elastic.co/t/metricbeat-data-not-flowing-into-elasticsearch/143328 "2018-08-08T13:13:47Z")
Hi All, Got into a small trouble, have installed metric beat in few VM's which was sending data to elasticsearch and was able to view all the dashboards properly, and the VM firewall was turned off. There was a vulnera… --- ## [How to drop the fields that Filebeat normally adds (type, source, offset, etc.)](https://discuss.elastic.co/t/how-to-drop-the-fields-that-filebeat-normally-adds-type-source-offset-etc/143235)
**Author:** [@Junble](https://discuss.elastic.co/u/Junble)\ **Replies:** 6\ **Last updated:** [August 8, 2018, 11:19am UTC](https://discuss.elastic.co/t/how-to-drop-the-fields-that-filebeat-normally-adds-type-source-offset-etc/143235 "2018-08-08T11:19:02Z")
Hi all As the title says,but i didn't find the setting about it! --- ## [Add local machine time field](https://discuss.elastic.co/t/add-local-machine-time-field/143441)
**Author:** [@Priyaranjan\_Mudliar](https://discuss.elastic.co/u/Priyaranjan_Mudliar)\ **Replies:** 3\ **Last updated:** [August 8, 2018, 11:13am UTC](https://discuss.elastic.co/t/add-local-machine-time-field/143441 "2018-08-08T11:13:11Z")
How should i add a new local machine time field to my filebeat output. I don't want the @timestamp because its in UTC. Please help me on this one. --- ## [Filebeat monitoring and sending logs to Elasticsearch](https://discuss.elastic.co/t/filebeat-monitoring-and-sending-logs-to-elasticsearch/143242)
**Author:** [@Priyaranjan\_Mudliar](https://discuss.elastic.co/u/Priyaranjan_Mudliar)\ **Replies:** 5\ **Last updated:** [August 8, 2018, 10:22am UTC](https://discuss.elastic.co/t/filebeat-monitoring-and-sending-logs-to-elasticsearch/143242 "2018-08-08T10:22:38Z")
Is it possible to run a batch script when i encounter a "Error" word in any of the log lines which is monitored by filebeat and is sent to elasticsearch. Can someone please help me on this? Any leads would be appreciate… --- ## [Reading multiple files and writing into multiple files](https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150)
**Author:** [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Replies:** 3\ **Last updated:** [August 8, 2018, 9:50am UTC](https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150 "2018-08-08T09:50:17Z")
I want to read multiple log files and write into multiple files (intention is to test). My Filebeat configuration looks like below : filebeat.prospectors: - input\_type: log enabled: true paths: - C:\\data\\Logs\\e… --- ## [Getsockopt: connection timed out](https://discuss.elastic.co/t/getsockopt-connection-timed-out/143315)
**Author:** [@Sam\_WCL](https://discuss.elastic.co/u/Sam_WCL)\ **Replies:** 4\ **Last updated:** [August 8, 2018, 7:34am UTC](https://discuss.elastic.co/t/getsockopt-connection-timed-out/143315 "2018-08-08T07:34:21Z")
Hello, I am having an issue on a few boxes reporting to my elastic stack, They look to be getting blocked/timing out with the below error in the metricbeat logs, 2018-08-07T12:33:49.810+0100 ERROR pipeline/output… --- ## [Beats via Logstash and Elasticsearch](https://discuss.elastic.co/t/beats-via-logstash-and-elasticsearch/142460)
**Author:** [@Sanjeev\_Gopal](https://discuss.elastic.co/u/Sanjeev_Gopal)\ **Replies:** 2\ **Last updated:** [August 8, 2018, 7:07am UTC](https://discuss.elastic.co/t/beats-via-logstash-and-elasticsearch/142460 "2018-08-08T07:07:56Z")
Reference: Conflicted about Beats, Logstash and ES Lets say I pipe a few Beats (in a remote DC) through Logstash, and a few beats direct to ES, can I send them to the same index? Wondering if Logstash would change the e… --- ## [About the time spent when output to elasticsearch](https://discuss.elastic.co/t/about-the-time-spent-when-output-to-elasticsearch/143290)
**Author:** [@cythwell](https://discuss.elastic.co/u/cythwell)\ **Replies:** 6\ **Last updated:** [August 8, 2018, 7:05am UTC](https://discuss.elastic.co/t/about-the-time-spent-when-output-to-elasticsearch/143290 "2018-08-08T07:05:34Z")
Hi, I am trying filebeat output to elasticsearch. And have something not clear. And I found that there are almost 7 seconds among filebeat publish to elasticsearch and elasticsearch API can find the doc. (Filebeat jus… --- ## [Multiple inputs](https://discuss.elastic.co/t/multiple-inputs/143102)
**Author:** [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Replies:** 1\ **Last updated:** [August 8, 2018, 6:35am UTC](https://discuss.elastic.co/t/multiple-inputs/143102 "2018-08-08T06:35:12Z")
I am testing filebeat to see if it can read multiple files based on log\_type i can make decision as to which index the logs have to go in elasticsearch from logstash. filebeat.prospectors: - input\_type: log enabled: … --- ## [Packetbeat local usage without elastic search](https://discuss.elastic.co/t/packetbeat-local-usage-without-elastic-search/141981)
**Author:** [@acsarnaik](https://discuss.elastic.co/u/acsarnaik)\ **Replies:** 3\ **Last updated:** [August 7, 2018, 10:17pm UTC](https://discuss.elastic.co/t/packetbeat-local-usage-without-elastic-search/141981 "2018-08-07T22:17:52Z")
Can packetbeat support local reporting of packets that are captured for a particular protocol, such as DNS, DHCP? Or does it have APIs to get the packet information such as DNS options without installing elasticsearch an… --- ## [Displaying the Full Exception and Caused By Message](https://discuss.elastic.co/t/displaying-the-full-exception-and-caused-by-message/143364)
**Author:** [@Luis\_Alejandro\_Galan](https://discuss.elastic.co/u/Luis_Alejandro_Galan)\ **Replies:** 4\ **Last updated:** [August 7, 2018, 8:38pm UTC](https://discuss.elastic.co/t/displaying-the-full-exception-and-caused-by-message/143364 "2018-08-07T20:38:28Z")
Hello, I am working on displaying the full error, exception and "caused by" portions of my logs. I used the following doc to install the ELK stack: My error example: 2018-03-01 10:24:20.014 ERROR 32649 --- \[Camel (c… --- ## [Filebeat and grok parsing errors](https://discuss.elastic.co/t/filebeat-and-grok-parsing-errors/143371)
**Author:** [@rdesanno](https://discuss.elastic.co/u/rdesanno)\ **Replies:** 4\ **Last updated:** [August 7, 2018, 6:47pm UTC](https://discuss.elastic.co/t/filebeat-and-grok-parsing-errors/143371 "2018-08-07T18:47:00Z")
This one should be pretty easy.... In the past, I was using filebeat to log to logstash directly, and putting in my own custom grok statements. It worked well but to be honest, was way more complicated than it needed to… --- ## [Beats monitoring through X pack](https://discuss.elastic.co/t/beats-monitoring-through-x-pack/143366)
**Author:** [@chandra0651](https://discuss.elastic.co/u/chandra0651)\ **Replies:** 2\ **Last updated:** [August 7, 2018, 5:39pm UTC](https://discuss.elastic.co/t/beats-monitoring-through-x-pack/143366 "2018-08-07T17:39:14Z")
Hi i want to monitor the file beat agents same like how i am monitoring the elastic,kibana and logstash through X pack . My ELK stack version is 6.2.1 which doesn't have beats\_user user and bests\_system role. Can any one… --- ## [Filebeat running a .bat file](https://discuss.elastic.co/t/filebeat-running-a-bat-file/143092)
**Author:** [@Priyaranjan\_Mudliar](https://discuss.elastic.co/u/Priyaranjan_Mudliar)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 5:27pm UTC](https://discuss.elastic.co/t/filebeat-running-a-bat-file/143092 "2018-08-07T17:27:44Z")
Can i somehow execute a bat file if my filebeat gets a log line which has a string "Error" in it ? Or can i output only those log lines to elasticsearch which has a string "Error"? Please help me on this one... --- ## [Add syslog output mode to filebeat error, which occurs at transport/client.go](https://discuss.elastic.co/t/add-syslog-output-mode-to-filebeat-error-which-occurs-at-transport-client-go/143228)
**Author:** [@changgq](https://discuss.elastic.co/u/changgq)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 4:43pm UTC](https://discuss.elastic.co/t/add-syslog-output-mode-to-filebeat-error-which-occurs-at-transport-client-go/143228 "2018-08-07T16:43:23Z")
Filebeat Version: 7.0.0-alpha1 ERROR: unsupported network type udp. source: libbeat/outputs/transport/client.go line 76. I want add syslog output to filebeat, but use udp to send messages error. I think this should… --- ## [Auditbeat fatal error (crash)](https://discuss.elastic.co/t/auditbeat-fatal-error-crash/143300)
**Author:** [@ppanagiotis](https://discuss.elastic.co/u/ppanagiotis)\ **Replies:** 3\ **Last updated:** [August 7, 2018, 4:00pm UTC](https://discuss.elastic.co/t/auditbeat-fatal-error-crash/143300 "2018-08-07T16:00:52Z")
auditbeat occasionally crashes with the following error: Aug 07 12:32:14 hostname auditbeat\[10686\]: fatal error: concurrent map writesA - Pastebin.com The package is build with master branch (head commit b84e083625ec… --- ## [Documentation on adding new Packetbeat protocols](https://discuss.elastic.co/t/documentation-on-adding-new-packetbeat-protocols/141831)
**Author:** [@dhughes](https://discuss.elastic.co/u/dhughes)\ **Replies:** 5\ **Last updated:** [August 7, 2018, 3:37pm UTC](https://discuss.elastic.co/t/documentation-on-adding-new-packetbeat-protocols/141831 "2018-08-07T15:37:17Z")
I'm looking for any documentation on the current packetbeat structure, as well as how to add new protocols ... if this exists. I've installed the project, compiled and tested a little. But other than just starting goin… --- ## [Using filebeat and add job name](https://discuss.elastic.co/t/using-filebeat-and-add-job-name/143238)
**Author:** [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 3:04pm UTC](https://discuss.elastic.co/t/using-filebeat-and-add-job-name/143238 "2018-08-07T15:04:56Z")
I have the below log structure of jobs: /job1/datetoday.log /job2/datetoday.log and so on. Can file beat be used to grep the error in logs and add Job name string to that error log fetched before indexing to ES so t… --- ## [How to build beats in windows](https://discuss.elastic.co/t/how-to-build-beats-in-windows/141711)
**Author:** [@Manjukb](https://discuss.elastic.co/u/Manjukb)\ **Replies:** 15\ **Last updated:** [August 7, 2018, 2:19pm UTC](https://discuss.elastic.co/t/how-to-build-beats-in-windows/141711 "2018-08-07T14:19:42Z")
i'm currently trying to build a beats plugin to get some windows specific info so trying to build it on windows seams there is lot of dependency on Linux to develop the beats if i try to do make setup it says mkdir c… --- ## [Multiple input configuration](https://discuss.elastic.co/t/multiple-input-configuration/143299)
**Author:** [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Replies:** 8\ **Last updated:** [August 7, 2018, 1:16pm UTC](https://discuss.elastic.co/t/multiple-input-configuration/143299 "2018-08-07T13:16:34Z")
I want to read multiple log files and send it to logstash. Below is my configuration (I just wanted to test it locally so the configuration I used is given below): filebeat.config.modules: path: ${path.config}/modules… --- ## [Multiple file inputs as sources](https://discuss.elastic.co/t/multiple-file-inputs-as-sources/143317)
**Author:** [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Replies:** 1\ **Last updated:** [August 7, 2018, 1:12pm UTC](https://discuss.elastic.co/t/multiple-file-inputs-as-sources/143317 "2018-08-07T13:12:12Z")
I want to ingest 2 different log files from filebeat to logstash. I have the below configuration for filebeat. I am using 6.3 version of ELK so don't worry about the path of file. Filebeat will take C:\\data\\log\\eis.log. … [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=428) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=430)