# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=360 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 361 --- ## [Parse filename before sending to ElasticSearch](https://discuss.elastic.co/t/parse-filename-before-sending-to-elasticsearch/177670)
**Author:** [@rahulnathan](https://discuss.elastic.co/u/rahulnathan)\ **Replies:** 8\ **Last updated:** [April 30, 2019, 5:20am UTC](https://discuss.elastic.co/t/parse-filename-before-sending-to-elasticsearch/177670 "2019-04-30T05:20:13Z")
I am trying to parse the filename to extract certain information. Is this possible today or need to send the data to LogStash to process further ? Thanks, Rahul --- ## [Filebeat 6.6.2 and autodiscover docker log harversting](https://discuss.elastic.co/t/filebeat-6-6-2-and-autodiscover-docker-log-harversting/175267)
**Author:** [@bmmpt](https://discuss.elastic.co/u/bmmpt)\ **Replies:** 2\ **Last updated:** [April 30, 2019, 4:32am UTC](https://discuss.elastic.co/t/filebeat-6-6-2-and-autodiscover-docker-log-harversting/175267 "2019-04-30T04:32:08Z")
On a Windows 10 OS, I have a series of docker containers running java applications. Each container internally logs to: /usr/app/logs/some\_log\_file\_name.log A sample log entry would look like: {"timestamp":"2019-04-02T… --- ## [Create custom metricbeat fields](https://discuss.elastic.co/t/create-custom-metricbeat-fields/178757)
**Author:** [@mladen](https://discuss.elastic.co/u/mladen)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 11:12pm UTC](https://discuss.elastic.co/t/create-custom-metricbeat-fields/178757 "2019-04-29T23:12:04Z")
Hello, For purpose of monitoring CPU utilization per service I have created custom metricbeat fields. Using metricbeat all data are transferred to logstash pipeline. This is my filter section: filter { if \[system\]\[p… --- ## [Filebeat publishes old event of log so I can't see real time log in kibana](https://discuss.elastic.co/t/filebeat-publishes-old-event-of-log-so-i-cant-see-real-time-log-in-kibana/178990)
**Author:** [@Jaepyoung\_Kim](https://discuss.elastic.co/u/Jaepyoung_Kim)\ **Replies:** 0\ **Last updated:** [April 29, 2019, 9:03pm UTC](https://discuss.elastic.co/t/filebeat-publishes-old-event-of-log-so-i-cant-see-real-time-log-in-kibana/178990 "2019-04-29T21:03:40Z")
The symptom is that I can't see the real time log in kibana. I changed filebeat to debug mode, and I found out that the filebeat publishes old log event. If you see my log below, filebeat publishes the log of April 29… --- ## [Heartbeat 7 configuration issues: monitors & index name](https://discuss.elastic.co/t/heartbeat-7-configuration-issues-monitors-index-name/178938)
**Author:** [@Marc-Antoine\_J](https://discuss.elastic.co/u/Marc-Antoine_J)\ **Replies:** 3\ **Last updated:** [April 29, 2019, 7:05pm UTC](https://discuss.elastic.co/t/heartbeat-7-configuration-issues-monitors-index-name/178938 "2019-04-29T19:05:24Z")
Hi all, I upgraded my full stack to 7.0 lately. This morning I noticed no heartbeat docs were coming in my elasticsearch. I started heartbeat with debug logs then I noticed it was not loading any monitors config files e… --- ## [Filebeat - Parse json output](https://discuss.elastic.co/t/filebeat-parse-json-output/178079)
**Author:** [@arianmotamedi](https://discuss.elastic.co/u/arianmotamedi)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 5:52pm UTC](https://discuss.elastic.co/t/filebeat-parse-json-output/178079 "2019-04-29T17:52:41Z")
I realize this has been asked numerous times but I'm having a hard time getting Filebeat 6.2.4 to send json logs to Elasticsearch (without Logstash) in a K8s environment. For example for a given json log: {"@timestamp"… --- ## [Filesystem stats not including xfs device](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703)
**Author:** [@Eric\_Herbrandson](https://discuss.elastic.co/u/Eric_Herbrandson)\ **Replies:** 10\ **Last updated:** [April 29, 2019, 4:52pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703 "2019-04-29T16:52:56Z")
I have 2 devices I expect to see logs for "xvda" and "nvme0n1". The second doesn't appear in kibana. It is a xfs filesystem. I do see one error showing up repeatedly in the metricbeat logs, but it's hard to tell if it's … --- ## [Hearbeat and elasticsearch : x509: certificate signed by unknown authority](https://discuss.elastic.co/t/hearbeat-and-elasticsearch-x509-certificate-signed-by-unknown-authority/178865)
**Author:** [@meddreamer](https://discuss.elastic.co/u/meddreamer)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 3:20pm UTC](https://discuss.elastic.co/t/hearbeat-and-elasticsearch-x509-certificate-signed-by-unknown-authority/178865 "2019-04-29T15:20:54Z")
Hi, I am trying to configure heartbeat with elasticsearch and kibana. I am using version 6.7.1 I am receiving this error: 2019-04-29T11:34:10.127+0300 ERROR instance/beat.go:906 Exiting: fail to create the Elasticse… --- ## [Filebeat 6.7.1: template spec](https://discuss.elastic.co/t/filebeat-6-7-1-template-spec/178954)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [April 29, 2019, 2:29pm UTC](https://discuss.elastic.co/t/filebeat-6-7-1-template-spec/178954 "2019-04-29T14:29:08Z")
Hi, I'm looking at the fields defined in fields.yml and see the following: - name: took\_millis description: "Time took in milliseconds" example: 42 type: keyword Is this deliberate? Why is type… --- ## [Tags in Filebeat 7.0 - FIXED](https://discuss.elastic.co/t/tags-in-filebeat-7-0-fixed/178704)
**Author:** [@antawari](https://discuss.elastic.co/u/antawari)\ **Replies:** 1\ **Last updated:** [April 29, 2019, 1:06pm UTC](https://discuss.elastic.co/t/tags-in-filebeat-7-0-fixed/178704 "2019-04-29T13:06:30Z")
Hello community, Here lies my problem: so far I have been able to parse several logs from my servers into logstash using Filbeat, the issue is that I need to insert tags into each path so I'm able to then show it in dif… --- ## [Create new beat as a service](https://discuss.elastic.co/t/create-new-beat-as-a-service/178933)
**Author:** [@Shani\_Berrebi](https://discuss.elastic.co/u/Shani_Berrebi)\ **Replies:** 0\ **Last updated:** [April 29, 2019, 1:03pm UTC](https://discuss.elastic.co/t/create-new-beat-as-a-service/178933 "2019-04-29T13:03:01Z")
hi, i created a new beat of my own for windows environment. it is created as an .exe file. how can i create from this beat a service on that runs on windows? thanks, Shani --- ## [Disable Docker Support?](https://discuss.elastic.co/t/disable-docker-support/178228)
**Author:** [@Helflym](https://discuss.elastic.co/u/Helflym)\ **Replies:** 10\ **Last updated:** [April 29, 2019, 12:17pm UTC](https://discuss.elastic.co/t/disable-docker-support/178228 "2019-04-29T12:17:14Z")
Hi, I'm trying to make beats/filebeat and maybe the other beats' packages available on AIX. I've managed to make it locally by porting elastic/go-txfile (I'll submit it soon) and by modifying vendor packages manually (… --- ## [We are not receiving the logs from filebeat to kafka](https://discuss.elastic.co/t/we-are-not-receiving-the-logs-from-filebeat-to-kafka/178756)
**Author:** [@saravananveera](https://discuss.elastic.co/u/saravananveera)\ **Replies:** 5\ **Last updated:** [April 29, 2019, 11:13am UTC](https://discuss.elastic.co/t/we-are-not-receiving-the-logs-from-filebeat-to-kafka/178756 "2019-04-29T11:13:17Z")
Hi We are using kafka 2.12 and configure the output as below in the filebeat. service versions: logstash-6.3.1 kafka\_2.12-2.2.0 zookeeper-3.4.14 elasticsearch-6.3.1 kibana-6.3.1 filebeat-6.3.1 filebeat.prospectors: … --- ## [Logstash / Filebeat not process all lines from error logs](https://discuss.elastic.co/t/logstash-filebeat-not-process-all-lines-from-error-logs/178670)
**Author:** [@John\_Doe3](https://discuss.elastic.co/u/John_Doe3)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 9:19am UTC](https://discuss.elastic.co/t/logstash-filebeat-not-process-all-lines-from-error-logs/178670 "2019-04-29T09:19:59Z")
Hey I am using newest ELK stack (V7) on Ubuntu 16.04 to process my log files from several nginx-servers. All the access-logs are working pretty fine. Whats not working is the error-log. Only 1% of all lines are visibil… --- ## [FILEBEAT REGULAR EXPRESSION FAILING IN MULTI LINE PATTERN](https://discuss.elastic.co/t/filebeat-regular-expression-failing-in-multi-line-pattern/178494)
**Author:** [@bomba](https://discuss.elastic.co/u/bomba)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 9:27am UTC](https://discuss.elastic.co/t/filebeat-regular-expression-failing-in-multi-line-pattern/178494 "2019-04-29T09:27:47Z")
We are using multiline pattern for file beats to parse the application logs. Multiline patterns is not able to merge related line as one log PS : We are using https://play.golang.org/ for testing. We are not able to … --- ## [Unable to view Filebeat index in Kibana](https://discuss.elastic.co/t/unable-to-view-filebeat-index-in-kibana/178836)
**Author:** [@vbrundavanam](https://discuss.elastic.co/u/vbrundavanam)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 9:19am UTC](https://discuss.elastic.co/t/unable-to-view-filebeat-index-in-kibana/178836 "2019-04-29T09:19:21Z")
Hi, I configured filebeat and winlogbeat on my windows VM. I am able to see winlogbeat and i cant see filebeat. Please suggest on this. Below are the details. Elasticsearch version : 6.3.2 Logstash version : 6.3.2 Ki… --- ## [Beats 7.0 won't start if "setup.dashboards.enabled = true" and kibana is not reachable](https://discuss.elastic.co/t/beats-7-0-wont-start-if-setup-dashboards-enabled-true-and-kibana-is-not-reachable/178875)
**Author:** [@stefan7018](https://discuss.elastic.co/u/stefan7018)\ **Replies:** 0\ **Last updated:** [April 29, 2019, 9:08am UTC](https://discuss.elastic.co/t/beats-7-0-wont-start-if-setup-dashboards-enabled-true-and-kibana-is-not-reachable/178875 "2019-04-29T09:08:10Z")
When in \*beat config the option "setup.dashboards.enabled = true" is set and the Kibana server is not reachable/or in startup phase while starting beats the beats startup will fail. Apr 29 08:15:26 elastic1 auditbeat: … --- ## [Hearbeat error: x509: certificate signed by unknown authority](https://discuss.elastic.co/t/hearbeat-error-x509-certificate-signed-by-unknown-authority/176484)
**Author:** [@Peter92](https://discuss.elastic.co/u/Peter92)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 8:28am UTC](https://discuss.elastic.co/t/hearbeat-error-x509-certificate-signed-by-unknown-authority/176484 "2019-04-29T08:28:00Z")
Hi Team, I am getting x509: certificate signed by unknown authority on heartbeat, although I have ssl.verification\_mode: none line in the config file. - type: http proxy\_url: ssl.verification\_mode: none # List… --- ## [FileBeats -Are there any ways we can delete the log files after file beat harvest the data to logstash](https://discuss.elastic.co/t/filebeats-are-there-any-ways-we-can-delete-the-log-files-after-file-beat-harvest-the-data-to-logstash/177997)
**Author:** [@prabhat\_ranjan](https://discuss.elastic.co/u/prabhat_ranjan)\ **Replies:** 5\ **Last updated:** [April 29, 2019, 7:35am UTC](https://discuss.elastic.co/t/filebeats-are-there-any-ways-we-can-delete-the-log-files-after-file-beat-harvest-the-data-to-logstash/177997 "2019-04-29T07:35:41Z")
FileBeats -Are there any ways we can delete the log files after file beat harvest the data to logstash. I am using window machine. I have json files(multiple) which is having application analyticsinfo's. I am reading th… --- ## [Packetbeat and Elasticsearch](https://discuss.elastic.co/t/packetbeat-and-elasticsearch/177768)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 6:54am UTC](https://discuss.elastic.co/t/packetbeat-and-elasticsearch/177768 "2019-04-29T06:54:29Z")
Hi, When will packbeat be able to monitor the elasticsearch transport interface (9300)? Regards, D --- ## [WinScp files issue with filebeat](https://discuss.elastic.co/t/winscp-files-issue-with-filebeat/178787)
**Author:** [@moti.mor](https://discuss.elastic.co/u/moti.mor)\ **Replies:** 2\ **Last updated:** [April 29, 2019, 5:15am UTC](https://discuss.elastic.co/t/winscp-files-issue-with-filebeat/178787 "2019-04-29T05:15:03Z")
Hi Guys, I have a strange issue, i have server centos with filebeat installed when i'm creating a file-log on the server all works good i see the file on kibana. but when i'm creating the file via winscp it's not worki… --- ## [Suricata module not working as expected](https://discuss.elastic.co/t/suricata-module-not-working-as-expected/175734)
**Author:** [@Greg\_1990](https://discuss.elastic.co/u/Greg_1990)\ **Replies:** 7\ **Last updated:** [April 29, 2019, 1:44am UTC](https://discuss.elastic.co/t/suricata-module-not-working-as-expected/175734 "2019-04-29T01:44:14Z")
I'm trying to use the Filebeat's Suricata module to send logs to Logstash (and then to Kibana) but I'm not receiving them in the correct format. All fields are hava "json." before them. And the logs are tagged as "beats\_… --- ## [Mutliline logs](https://discuss.elastic.co/t/mutliline-logs/178786)
**Author:** [@Ameed\_Ashour](https://discuss.elastic.co/u/Ameed_Ashour)\ **Replies:** 0\ **Last updated:** [April 28, 2019, 7:05am UTC](https://discuss.elastic.co/t/mutliline-logs/178786 "2019-04-28T07:05:45Z")
Hi, I am trying to do multiline in this lines: 2019-03-07 10:00:32.853 29 ERROR nova.conductor.manager Traceback (most recent call last): 2019-03-07 10:00:32.853 29 ERROR nova.conductor.manager 2019-03-07 10:00:32… --- ## [2 instances of packetbeat on the same server but on kibana monitor i see only 1](https://discuss.elastic.co/t/2-instances-of-packetbeat-on-the-same-server-but-on-kibana-monitor-i-see-only-1/178441)
**Author:** [@Itay\_Rozenberg](https://discuss.elastic.co/u/Itay_Rozenberg)\ **Replies:** 2\ **Last updated:** [April 27, 2019, 6:45pm UTC](https://discuss.elastic.co/t/2-instances-of-packetbeat-on-the-same-server-but-on-kibana-monitor-i-see-only-1/178441 "2019-04-27T18:45:53Z")
hi guys i have 2 instances of packetbeat on the same server. i'v enabled "xpack.monitoring.enabled" on both of them. on kibana monitoring i only see one of them, if i disable one of them i see still see one beat on k… --- ## [RE: /usr/local/bin/docker-entrypoint: line 8: exec: filebeat: not found](https://discuss.elastic.co/t/re-usr-local-bin-docker-entrypoint-line-8-exec-filebeat-not-found/177648)
**Author:** [@Michele\_Chersich](https://discuss.elastic.co/u/Michele_Chersich)\ **Replies:** 3\ **Last updated:** [April 26, 2019, 5:14pm UTC](https://discuss.elastic.co/t/re-usr-local-bin-docker-entrypoint-line-8-exec-filebeat-not-found/177648 "2019-04-26T17:14:01Z")
I basically have the same problem as this other topic, but I am reposting it since no solution was found. Deploying filebeats via docker-compose produces the following error: /usr/local/bin/docker-entrypoint: line 8: e… --- ## [TCP input message size problem, maybe a bug](https://discuss.elastic.co/t/tcp-input-message-size-problem-maybe-a-bug/178615)
**Author:** [@SuperChia](https://discuss.elastic.co/u/SuperChia)\ **Replies:** 2\ **Last updated:** [April 27, 2019, 3:08am UTC](https://discuss.elastic.co/t/tcp-input-message-size-problem-maybe-a-bug/178615 "2019-04-27T03:08:43Z")
When Filebeat uses “TCP input”, you can set the option "max\_message\_size" as maximum size in bytes of the message received over TCP. But, there is another option that limits the message size when Filebeat calls "scanner… --- ## [Filebeat can't connect to Kibana(Docker)](https://discuss.elastic.co/t/filebeat-cant-connect-to-kibana-docker/178556)
**Author:** [@croatech](https://discuss.elastic.co/u/croatech)\ **Replies:** 0\ **Last updated:** [April 26, 2019, 3:11am UTC](https://discuss.elastic.co/t/filebeat-cant-connect-to-kibana-docker/178556 "2019-04-26T03:11:22Z")
All versions are 7.0.0 I've got an error when I'm trying to setup filebeat: Loading dashboards (Kibana must be running and reachable) Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET reques… --- ## [Metricbeat on 10 hosts - how to config for low shards](https://discuss.elastic.co/t/metricbeat-on-10-hosts-how-to-config-for-low-shards/176982)
**Author:** [@Sushimaster](https://discuss.elastic.co/u/Sushimaster)\ **Replies:** 9\ **Last updated:** [April 26, 2019, 9:25pm UTC](https://discuss.elastic.co/t/metricbeat-on-10-hosts-how-to-config-for-low-shards/176982 "2019-04-26T21:25:03Z")
I'm running metricbeat on around 10 hosts. In ES each host is generating each day a shard or index. After upgrade to ES7 kibana throws an error like Validation Failed: 1: this action would add \[2\] total shards, but t… --- ## [Filebeat 5.6.16 modifies logstash URL:PORT in runtime](https://discuss.elastic.co/t/filebeat-5-6-16-modifies-logstash-url-port-in-runtime/178663)
**Author:** [@dumkaz](https://discuss.elastic.co/u/dumkaz)\ **Replies:** 1\ **Last updated:** [April 26, 2019, 9:18pm UTC](https://discuss.elastic.co/t/filebeat-5-6-16-modifies-logstash-url-port-in-runtime/178663 "2019-04-26T21:18:14Z")
Hi I am running filebeat 5.6.16 on Ubuntu 18.04 x64 Here is my filebeat output section configuration: output.logstash: Blockquote hosts: \["glog-tcp-1480968407.us-east-1.elb.amazonaws.com:12904"\] bulk\_max\_size: 100… --- ## [How does -E flag works](https://discuss.elastic.co/t/how-does-e-flag-works/178530)
**Author:** [@Sebastian\_Treu](https://discuss.elastic.co/u/Sebastian_Treu)\ **Replies:** 2\ **Last updated:** [April 26, 2019, 5:23pm UTC](https://discuss.elastic.co/t/how-does-e-flag-works/178530 "2019-04-26T17:23:49Z")
Hi, I'm trying to overwrite the output of filebeat. Motivation is based on my idea of avoid at all cost mounting specific files into docker container, so I though of: ~$ docker run --user root -it -v "/var/lib/docker/… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=359) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=361)