# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=508 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 509 --- ## [Cant get regex to exclude line to work](https://discuss.elastic.co/t/cant-get-regex-to-exclude-line-to-work/95432)
**Author:** [@Puk](https://discuss.elastic.co/u/Puk)\ **Replies:** 5\ **Last updated:** [August 3, 2017, 2:33am UTC](https://discuss.elastic.co/t/cant-get-regex-to-exclude-line-to-work/95432 "2017-08-03T02:33:16Z")
Newbie to ELK Stack and Filebeat here. Managed to get log shipping from my linux hosts working fine and have 2x prospectors for my syslogs and secure logs as going to try filtering them in logstash differently. Anyway,… --- ## [Consumer Groups not reporting offset topics](https://discuss.elastic.co/t/consumer-groups-not-reporting-offset-topics/95364)
**Author:** [@oleksandrtub](https://discuss.elastic.co/u/oleksandrtub)\ **Replies:** 3\ **Last updated:** [August 2, 2017, 7:44pm UTC](https://discuss.elastic.co/t/consumer-groups-not-reporting-offset-topics/95364 "2017-08-02T19:44:23Z")
Hi All, Got a problem with metricbeat kafka consumer groups offset for topics/partition. After updating Kafka from 0.10.0.1 to 0.10.2.1, stoped reporting offset. Bellow part of debug log related to kafka module: 2017… --- ## [Quoted json log message](https://discuss.elastic.co/t/quoted-json-log-message/95398)
**Author:** [@grants](https://discuss.elastic.co/u/grants)\ **Replies:** 2\ **Last updated:** [August 2, 2017, 7:23pm UTC](https://discuss.elastic.co/t/quoted-json-log-message/95398 "2017-08-02T19:23:39Z")
I have some event log files that are written in json, but they are wrapped in quotes. Filebeats is dumping errors like this: 2017/08/01 18:37:12.103066 json.go:34: ERR Error decoding JSON: json: cannot unmarshal string… --- ## [MongoDB Replica Set Metricset](https://discuss.elastic.co/t/mongodb-replica-set-metricset/95588)
**Author:** [@ShanTulshi](https://discuss.elastic.co/u/ShanTulshi)\ **Replies:** 0\ **Last updated:** [August 2, 2017, 5:33pm UTC](https://discuss.elastic.co/t/mongodb-replica-set-metricset/95588 "2017-08-02T17:33:59Z")
I’ve been searching for a little while for the best way to collect MongoDB replica set metrics, specifically those collected by the replSetGetStatus command. Is there support for this either now or planned for the future… --- ## [New Beat authentication settings](https://discuss.elastic.co/t/new-beat-authentication-settings/95540)
**Author:** [@berfinsari](https://discuss.elastic.co/u/berfinsari)\ **Replies:** 2\ **Last updated:** [August 2, 2017, 4:02pm UTC](https://discuss.elastic.co/t/new-beat-authentication-settings/95540 "2017-08-02T16:02:09Z")
hi, I am working on creating a beat for etcd. I can get the statistics but i have a question for authentication. If password and username are not set, "what should the beat do?" The beat gave an error message and it di… --- ## [Can Filebeat and Metricbeat together be using the same port of logstash](https://discuss.elastic.co/t/can-filebeat-and-metricbeat-together-be-using-the-same-port-of-logstash/93648)
**Author:** [@Raghunandan\_Sk](https://discuss.elastic.co/u/Raghunandan_Sk)\ **Replies:** 9\ **Last updated:** [August 2, 2017, 2:06pm UTC](https://discuss.elastic.co/t/can-filebeat-and-metricbeat-together-be-using-the-same-port-of-logstash/93648 "2017-08-02T14:06:07Z")
Hello , I am new to Metricbeat . I have system set which is running Filebeat ---\> logstash ----\> ES ----\> Kibana. Now I want to configure Metricbeat and want to send the data into logstash . So I stopped the filebeat and… --- ## [Log shipping question](https://discuss.elastic.co/t/log-shipping-question/95477)
**Author:** [@jarves](https://discuss.elastic.co/u/jarves)\ **Replies:** 1\ **Last updated:** [August 2, 2017, 10:24am UTC](https://discuss.elastic.co/t/log-shipping-question/95477 "2017-08-02T10:24:30Z")
Hi, I’m new to ELK. My setup is I have a dedicated server for ELK and I have two servers to which I will get the logs which are windows event logs. Here are my questions: Do I need to install a log shipper for each of… --- ## [Packetbeat & MySQL username](https://discuss.elastic.co/t/packetbeat-mysql-username/95446)
**Author:** [@tonybreak](https://discuss.elastic.co/u/tonybreak)\ **Replies:** 2\ **Last updated:** [August 2, 2017, 10:15am UTC](https://discuss.elastic.co/t/packetbeat-mysql-username/95446 "2017-08-02T10:15:03Z")
I found that the MySQL packetbeat has no username of database. I crawled network traffic to test, and found that there was no username in it. So, do you plan to add the username of database in MySQL packetbeat? --- ## [New module - fields.yml and filebeat.template.json](https://discuss.elastic.co/t/new-module-fields-yml-and-filebeat-template-json/95352)
**Author:** [@edisuc](https://discuss.elastic.co/u/edisuc)\ **Replies:** 2\ **Last updated:** [August 2, 2017, 10:07am UTC](https://discuss.elastic.co/t/new-module-fields-yml-and-filebeat-template-json/95352 "2017-08-02T10:07:00Z")
Hi. I’m creating a new module to parse proftpd’s xferlog log file. I got the parsing ok, but I’m stuck with Elastic Search mappings. I realized, that the mappings are done by uploading the template file filebeat.templat… --- ## [How to shut down filebeat properly](https://discuss.elastic.co/t/how-to-shut-down-filebeat-properly/95422)
**Author:** [@filebeater](https://discuss.elastic.co/u/filebeater)\ **Replies:** 1\ **Last updated:** [August 2, 2017, 8:47am UTC](https://discuss.elastic.co/t/how-to-shut-down-filebeat-properly/95422 "2017-08-02T08:47:53Z")
Hello, We run filebeat from the command line. I am wondering is there a way to shutdown filebeat properly other than “kill -9”, so that filebeat has chance to save its state before exit? thanks. yan --- ## [Filebeat show unwanted logs](https://discuss.elastic.co/t/filebeat-show-unwanted-logs/95179)
**Author:** [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Replies:** 10\ **Last updated:** [August 2, 2017, 7:36am UTC](https://discuss.elastic.co/t/filebeat-show-unwanted-logs/95179 "2017-08-02T07:36:10Z")
\#cat /etc/filebeat/filebeat.yml filebeat.modules: filebeat.prospectors: input\_type: log paths: /home/local/user/nsms/logs/Logs/nsms-server.log #include\_lines: \[“GP+”\] #- c:\\programdata\\elasticsearch\\logs\* outp… --- ## [Offcice 365 with beat?](https://discuss.elastic.co/t/offcice-365-with-beat/95267)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 5\ **Last updated:** [August 2, 2017, 3:01am UTC](https://discuss.elastic.co/t/offcice-365-with-beat/95267 "2017-08-02T03:01:13Z")
Can i get audit log form office 365 with awesome beat? Maybeat O365beat :smiley: Or have any solution for this case. Thanks :smiley: --- ## [Multiline codec in filebeats](https://discuss.elastic.co/t/multiline-codec-in-filebeats/95418)
**Author:** [@SandhyaRani](https://discuss.elastic.co/u/SandhyaRani)\ **Replies:** 2\ **Last updated:** [August 1, 2017, 9:48pm UTC](https://discuss.elastic.co/t/multiline-codec-in-filebeats/95418 "2017-08-01T21:48:19Z")
Hi there! I’m trying to use multiline codec in filebeat.yml for stack trace exceptions Here is my filebeat.conf - input\_type: log multiline.pattern: '^\[\[:space:\]\]' multiline.negate: false multiline.match: afte… --- ## [Is it possible to specify a partition key when using filebeat to send to a Kafka topic?](https://discuss.elastic.co/t/is-it-possible-to-specify-a-partition-key-when-using-filebeat-to-send-to-a-kafka-topic/94732)
**Author:** [@sw90000](https://discuss.elastic.co/u/sw90000)\ **Replies:** 6\ **Last updated:** [August 1, 2017, 4:54pm UTC](https://discuss.elastic.co/t/is-it-possible-to-specify-a-partition-key-when-using-filebeat-to-send-to-a-kafka-topic/94732 "2017-08-01T16:54:56Z")
I found a topic one years ago, it is not possible at that time. Can we do it now with filebeat 5.5.1? I want some part of the message to be the key of the kafka message. For example, the log message is like: 2017:07:21 i… --- ## [Log4J](https://discuss.elastic.co/t/log4j/95345)
**Author:** [@whgibbo](https://discuss.elastic.co/u/whgibbo)\ **Replies:** 1\ **Last updated:** [August 1, 2017, 2:14pm UTC](https://discuss.elastic.co/t/log4j/95345 "2017-08-01T14:14:45Z")
Hi, I'm new to using Filebeats/Logstash and have a application that is using Log4J and wanted to know what was the best option to allow for searching/filters based on the information in the log file from Kibana.. Here … --- ## [Failed to publish events](https://discuss.elastic.co/t/failed-to-publish-events/94983)
**Author:** [@pastperfect](https://discuss.elastic.co/u/pastperfect)\ **Replies:** 5\ **Last updated:** [August 1, 2017, 12:53pm UTC](https://discuss.elastic.co/t/failed-to-publish-events/94983 "2017-08-01T12:53:47Z")
I have set up packetbeat on a single client workstation alongside winlogbeat. When running packetbeat it doesn't seem to send the data through to my logstash server. These are the errors I get: ERR Failed to publish ev… --- ## [Filebeat reports :Error publishing events (retrying): read tcp IPA-\>9IPB5044: i/o timeout, reason?](https://discuss.elastic.co/t/filebeat-reports-error-publishing-events-retrying-read-tcp-ipa-9ipb5044-i-o-timeout-reason/95138)
**Author:** [@ndarkness](https://discuss.elastic.co/u/ndarkness)\ **Replies:** 5\ **Last updated:** [August 1, 2017, 12:15pm UTC](https://discuss.elastic.co/t/filebeat-reports-error-publishing-events-retrying-read-tcp-ipa-9ipb5044-i-o-timeout-reason/95138 "2017-08-01T12:15:24Z")
HEllo, I have updated my machine to filebeat 5 and now when trying to ship my logs I see that the shipping sometimes fails and I don’t understand the reason. The error is the following when enabling debug 2017-06-14T1… --- ## [Wrong phrase in the http.response.phrase field](https://discuss.elastic.co/t/wrong-phrase-in-the-http-response-phrase-field/95182)
**Author:** [@KBuev](https://discuss.elastic.co/u/KBuev)\ **Replies:** 3\ **Last updated:** [August 1, 2017, 12:05pm UTC](https://discuss.elastic.co/t/wrong-phrase-in-the-http-response-phrase-field/95182 "2017-08-01T12:05:03Z")
Hi, It appears that Packetbeat 5.5.0 (the HTTP module) only gets the last word from HTTP status line when searching for the response phrase. HTTP/1.1 500 Internal Server Error turns into “Error” in the http.response.ph… --- ## [Mongo db metric set 5.5.0 log errors](https://discuss.elastic.co/t/mongo-db-metric-set-5-5-0-log-errors/94333)
**Author:** [@randude](https://discuss.elastic.co/u/randude)\ **Replies:** 0\ **Last updated:** [July 24, 2017, 1:02pm UTC](https://discuss.elastic.co/t/mongo-db-metric-set-5-5-0-log-errors/94333 "2017-07-24T13:02:55Z")
I'm getting these messages: 2017-07-24 13:01:33 +0000 /usr/share/metricbeat/bin/metricbeat\[3031\]: INFO - - beat.go:207: metricbeat start running. 2017-07-24 13:01:33 +0000 /usr/share/metricbeat/bin/metricbeat\[3031\]: E… --- ## [How to improve Filebeat -\> Elastic performance & reduce Elastic store size ? got ~9433 logs per second](https://discuss.elastic.co/t/how-to-improve-filebeat-elastic-performance-reduce-elastic-store-size-got-9433-logs-per-second/95092)
**Author:** [@vramakrishnan](https://discuss.elastic.co/u/vramakrishnan)\ **Replies:** 4\ **Last updated:** [August 1, 2017, 8:22am UTC](https://discuss.elastic.co/t/how-to-improve-filebeat-elastic-performance-reduce-elastic-store-size-got-9433-logs-per-second/95092 "2017-08-01T08:22:27Z")
I have the following setup on MAC running on i7 (8core machine). Filebeat (reading local /tmp/perf/… logs) --\> Elastic single instance docker I was only able to get ~9400 logs/sec indexed to Elastic and also the store.… --- ## [Filebeat loadbalancer logstash scalability](https://discuss.elastic.co/t/filebeat-loadbalancer-logstash-scalability/95204)
**Author:** [@abhiroyg](https://discuss.elastic.co/u/abhiroyg)\ **Replies:** 2\ **Last updated:** [August 1, 2017, 7:36am UTC](https://discuss.elastic.co/t/filebeat-loadbalancer-logstash-scalability/95204 "2017-08-01T07:36:30Z")
Hi, If I use Filebeat internal loadbalancer, and say I want to add a new logstash node. I would have to take down all my filebeat instances and then change the configuration and start them up, right ? Is there another w… --- ## [Getting metricbeat process-related info only for some specified processes](https://discuss.elastic.co/t/getting-metricbeat-process-related-info-only-for-some-specified-processes/95115)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 2\ **Last updated:** [August 1, 2017, 5:26am UTC](https://discuss.elastic.co/t/getting-metricbeat-process-related-info-only-for-some-specified-processes/95115 "2017-08-01T05:26:55Z")
Hi, I want to configure my metricbeat.yml file in a way so that i can get process-related info(for example, system.process.pid,system.process.memory.size etc) where system.process.name = chrome.exe OR outlook.exe OR jav… --- ## [Additional field for winlogbeat](https://discuss.elastic.co/t/additional-field-for-winlogbeat/94750)
**Author:** [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\ **Replies:** 10\ **Last updated:** [August 1, 2017, 1:33am UTC](https://discuss.elastic.co/t/additional-field-for-winlogbeat/94750 "2017-08-01T01:33:25Z")
Is there a way that I can append data or field in winlog? Let's say I have a csv file which consists of 5 data. Can I append this everytime windows logs are sent thru beats? TIA --- ## [Can Filebeat access different logs](https://discuss.elastic.co/t/can-filebeat-access-different-logs/95125)
**Author:** [@abhisek](https://discuss.elastic.co/u/abhisek)\ **Replies:** 1\ **Last updated:** [August 1, 2017, 1:21am UTC](https://discuss.elastic.co/t/can-filebeat-access-different-logs/95125 "2017-08-01T01:21:20Z")
Hello, Can file-beat access different logs. By different i mean, i have an application which generate logs on a different server and i want to access it on the server where elk is installed. but i have used the defaul… --- ## [Is it possible to use Filebeat 6.0 with ES Cluster 5.x?](https://discuss.elastic.co/t/is-it-possible-to-use-filebeat-6-0-with-es-cluster-5-x/95220)
**Author:** [@outcoldman](https://discuss.elastic.co/u/outcoldman)\ **Replies:** 1\ **Last updated:** [August 1, 2017, 1:20am UTC](https://discuss.elastic.co/t/is-it-possible-to-use-filebeat-6-0-with-es-cluster-5-x/95220 "2017-08-01T01:20:06Z")
Filebeat has a lot of great features. Is the output of filebeat compatible with ES Cluster 5.x? Including mapping, kibana dashboards, etc? This page does not have 6.0 yet https://www.elastic.co/support/matrix#show\_compa… --- ## [Master branch docker image](https://discuss.elastic.co/t/master-branch-docker-image/95064)
**Author:** [@kvalev](https://discuss.elastic.co/u/kvalev)\ **Replies:** 2\ **Last updated:** [July 31, 2017, 3:54pm UTC](https://discuss.elastic.co/t/master-branch-docker-image/95064 "2017-07-31T15:54:03Z")
I am interested in running the latest metricbeat version, as it contains a new module that I need and I am looking for ways to create a docker image for the latest master branch state. As I understand you do not provide … --- ## [Multiline Pattern not Matching correctly](https://discuss.elastic.co/t/multiline-pattern-not-matching-correctly/94985)
**Author:** [@AlBytor](https://discuss.elastic.co/u/AlBytor)\ **Replies:** 4\ **Last updated:** [July 31, 2017, 3:35pm UTC](https://discuss.elastic.co/t/multiline-pattern-not-matching-correctly/94985 "2017-07-31T15:35:01Z")
My current Filebeat configuration is: But it keeps combining all of them together: As you can see my pattern consists of one letter and numbers: 2017-07-28T03:33:19.676-0500 Does anyone know what multiline.patter… --- ## [Configuring XML into ElasticSearch Using Filebeat](https://discuss.elastic.co/t/configuring-xml-into-elasticsearch-using-filebeat/95156)
**Author:** [@adikadashrieq](https://discuss.elastic.co/u/adikadashrieq)\ **Replies:** 4\ **Last updated:** [July 31, 2017, 9:49am UTC](https://discuss.elastic.co/t/configuring-xml-into-elasticsearch-using-filebeat/95156 "2017-07-31T09:49:40Z")
Hai Everyone, i want to transfer the list of file from xml into elasticsearch using Filebeat. I don’t know where to start its is because when i read the documentation be default filebeat load recommended files which in j… --- ## [Packetbeat, af\_packet and Linux 2.6](https://discuss.elastic.co/t/packetbeat-af-packet-and-linux-2-6/94640)
**Author:** [@KBuev](https://discuss.elastic.co/u/KBuev)\ **Replies:** 7\ **Last updated:** [July 31, 2017, 6:46am UTC](https://discuss.elastic.co/t/packetbeat-af-packet-and-linux-2-6/94640 "2017-07-31T06:46:20Z")
Hi, everyone. I'm having some trouble trying to setup Packetbeat on a Linux VM with RHEL 6.7 (Linux 2.6.32). If I use the following settings: packetbeat.interfaces.device: any packetbeat.interfaces.type: af\_packet … --- ## [Not able to send data to remote host](https://discuss.elastic.co/t/not-able-to-send-data-to-remote-host/91731)
**Author:** [@Prateek\_Kshtriya](https://discuss.elastic.co/u/Prateek_Kshtriya)\ **Replies:** 5\ **Last updated:** [July 28, 2017, 9:17am UTC](https://discuss.elastic.co/t/not-able-to-send-data-to-remote-host/91731 "2017-07-28T09:17:54Z")
Hi All, I am not able to post data to elastic-search remote host . Please suggest if there is any specific configuration required to be done to post data from metric-beat to different Elastic-search host. Below error… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=507) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=509)