# Beats **URL:** https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=402 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 403 --- ## [Understanding the role of fields.yml](https://discuss.elastic.co/t/understanding-the-role-of-fields-yml/157854)
**Author:** [@Martin\_H\_Andersen](https://discuss.elastic.co/u/Martin_H_Andersen)\ **Replies:** 1\ **Last updated:** [November 22, 2018, 1:30pm UTC](https://discuss.elastic.co/t/understanding-the-role-of-fields-yml/157854 "2018-11-22T13:30:24Z")
My setup is this: filebeat -\> logstash -\> elasticsearch I don't understand why filebeat is responsible for defining the fields mapping in elasticsearch. Fields can be altered in logstash? Giving new names. We have a v… --- ## [Docker + Persistent Registry = autodiscover failing](https://discuss.elastic.co/t/docker-persistent-registry-autodiscover-failing/157688)
**Author:** [@lifeofguenter](https://discuss.elastic.co/u/lifeofguenter)\ **Replies:** 4\ **Last updated:** [November 22, 2018, 1:14pm UTC](https://discuss.elastic.co/t/docker-persistent-registry-autodiscover-failing/157688 "2018-11-22T13:14:01Z")
Hi all, I would like to run filebeat as a docker container. The idea would be to volume mount the data folder so the registry can persistent between container (or even host) crashes/restarts. If I do so however, I get … --- ## [Adding fields does not work with docker input](https://discuss.elastic.co/t/adding-fields-does-not-work-with-docker-input/157852)
**Author:** [@lifeofguenter](https://discuss.elastic.co/u/lifeofguenter)\ **Replies:** 0\ **Last updated:** [November 22, 2018, 10:11am UTC](https://discuss.elastic.co/t/adding-fields-does-not-work-with-docker-input/157852 "2018-11-22T10:11:17Z")
filebeat.autodiscover: providers: - type: docker templates: - condition: has\_fields: - docker.container.labels.com.foobar.logs.type config: - type: dock… --- ## [Filebeat sends the data to the logstash once a second](https://discuss.elastic.co/t/filebeat-sends-the-data-to-the-logstash-once-a-second/157838)
**Author:** [@xodn0812](https://discuss.elastic.co/u/xodn0812)\ **Replies:** 1\ **Last updated:** [November 22, 2018, 9:52am UTC](https://discuss.elastic.co/t/filebeat-sends-the-data-to-the-logstash-once-a-second/157838 "2018-11-22T09:52:10Z")
filebeat sends the data to the logstash once a second. I want to change the 1 second unit. to 0.1 seconds. do you know how? --- ## [System.diskio obtaining NaN (Failed to serialize the event: unsupported float value: NaN)](https://discuss.elastic.co/t/system-diskio-obtaining-nan-failed-to-serialize-the-event-unsupported-float-value-nan/156877)
**Author:** [@niteman](https://discuss.elastic.co/u/niteman)\ **Replies:** 9\ **Last updated:** [November 21, 2018, 9:35pm UTC](https://discuss.elastic.co/t/system-diskio-obtaining-nan-failed-to-serialize-the-event-unsupported-float-value-nan/156877 "2018-11-21T21:35:31Z")
We're just setting up metricbeat and getting in a bunch of machines communication errors with logstash: |2018-11-15T12:01:59.051+0100|INFO|pipeline/output.go:95|Connecting to backoff(async(tcp://anonDEST\_HOST:anonDEST\_P… --- ## [Is there any way to define core number, filebeat should use?](https://discuss.elastic.co/t/is-there-any-way-to-define-core-number-filebeat-should-use/155041)
**Author:** [@antonm](https://discuss.elastic.co/u/antonm)\ **Replies:** 4\ **Last updated:** [November 21, 2018, 7:39pm UTC](https://discuss.elastic.co/t/is-there-any-way-to-define-core-number-filebeat-should-use/155041 "2018-11-21T19:39:32Z")
max\_procs Sets the maximum number of CPUs that can be executing simultaneously. The default is the number of logical CPUs available in the system. If i set "max\_procs: 1". Which core would be used ? Is there any way t… --- ## [Filebeat Line Endings Problem '\\n'](https://discuss.elastic.co/t/filebeat-line-endings-problem-n/157568)
**Author:** [@ahbe2901](https://discuss.elastic.co/u/ahbe2901)\ **Replies:** 5\ **Last updated:** [November 21, 2018, 3:48pm UTC](https://discuss.elastic.co/t/filebeat-line-endings-problem-n/157568 "2018-11-21T15:48:51Z")
I have the following setting: Filebeat =\> Logstash My problem is, that Filebeat is not recognizing \\n as a line ending and is then packing multiple lines into one message. Log Input (with Unix line endings \\n): 2018-1… --- ## [Filebeat configurations for multiline json input](https://discuss.elastic.co/t/filebeat-configurations-for-multiline-json-input/156992)
**Author:** [@Susmitha\_Devathi](https://discuss.elastic.co/u/Susmitha_Devathi)\ **Replies:** 5\ **Last updated:** [November 21, 2018, 2:49pm UTC](https://discuss.elastic.co/t/filebeat-configurations-for-multiline-json-input/156992 "2018-11-21T14:49:29Z")
Hi Team, could you please let me know how to write json filter and filebeat configurations for below data . { agentId: "TMS", apiVersion: "v2", entities: \[ { agentId: "susmitha", name: "EFGH", cacheManagerName: "Articl… --- ## [Environment variable error](https://discuss.elastic.co/t/environment-variable-error/156709)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 4\ **Last updated:** [November 21, 2018, 2:35pm UTC](https://discuss.elastic.co/t/environment-variable-error/156709 "2018-11-21T14:35:49Z")
Hi! There isn't too many documents or threads on this issue and I reckon it isn't used too much. I've several external configuration files to make updating more dynamic. I'd like to add environment variables to these w… --- ## [Filbeat on Kubernetes/Autodiscover stops sending logs](https://discuss.elastic.co/t/filbeat-on-kubernetes-autodiscover-stops-sending-logs/157571)
**Author:** [@fbcbarbosa](https://discuss.elastic.co/u/fbcbarbosa)\ **Replies:** 3\ **Last updated:** [November 21, 2018, 2:24pm UTC](https://discuss.elastic.co/t/filbeat-on-kubernetes-autodiscover-stops-sending-logs/157571 "2018-11-21T14:24:22Z")
Hi everyone, So we're having this problem for over a week now. We're running Filebeat on a Kubernetes cluster with Istio (not sure if relevant). The cluster has about ~50 nodes and ~700 pods. Everything worked fine for… --- ## [IIS Filebeat Module - Settings for Ignore Older?](https://discuss.elastic.co/t/iis-filebeat-module-settings-for-ignore-older/157186)
**Author:** [@Brett\_Larson](https://discuss.elastic.co/u/Brett_Larson)\ **Replies:** 1\ **Last updated:** [November 21, 2018, 2:13pm UTC](https://discuss.elastic.co/t/iis-filebeat-module-settings-for-ignore-older/157186 "2018-11-21T14:13:57Z")
Hello, Does the IIS module for Filebeat support the ignore\_older: setting? I'm unclear based on reading the documentation. Thank you! --- ## [Decoding json data under custom field](https://discuss.elastic.co/t/decoding-json-data-under-custom-field/157247)
**Author:** [@Vincehood](https://discuss.elastic.co/u/Vincehood)\ **Replies:** 2\ **Last updated:** [November 20, 2018, 7:30pm UTC](https://discuss.elastic.co/t/decoding-json-data-under-custom-field/157247 "2018-11-20T19:30:59Z")
Hello, let's say I have 2 software components which send json logs but with conflicting formats (for example, level is an integer for one and a string for the other). Is there any way I can use filebeat to decode json d… --- ## [Number of primary shards](https://discuss.elastic.co/t/number-of-primary-shards/157471)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 4\ **Last updated:** [November 21, 2018, 6:57am UTC](https://discuss.elastic.co/t/number-of-primary-shards/157471 "2018-11-21T06:57:04Z")
hi, I am using filebeat to ship logs into elasticsearch, in the filebeat.yml, we can define number of primary shards of created index. I am creating a weekly index as following: index: "myn-%{+yyyy.ww}" and the size… --- ## [How to send logs to logstash from filebeat in the order they are printed in the log file](https://discuss.elastic.co/t/how-to-send-logs-to-logstash-from-filebeat-in-the-order-they-are-printed-in-the-log-file/155512)
**Author:** [@Manoj\_Hettiarachchi](https://discuss.elastic.co/u/Manoj_Hettiarachchi)\ **Replies:** 6\ **Last updated:** [November 21, 2018, 3:57am UTC](https://discuss.elastic.co/t/how-to-send-logs-to-logstash-from-filebeat-in-the-order-they-are-printed-in-the-log-file/155512 "2018-11-21T03:57:51Z")
Description: In my system, I am using Filebeat, Logstash, Elasticsearch, and Kibana. For every transaction in the system, a log is printed in the log file and it is saved in the Elasticsearch db. There are around 200 … --- ## [How to understand "registry" file in filebeat](https://discuss.elastic.co/t/how-to-understand-registry-file-in-filebeat/157271)
**Author:** [@luxiaoxun](https://discuss.elastic.co/u/luxiaoxun)\ **Replies:** 2\ **Last updated:** [November 21, 2018, 2:36am UTC](https://discuss.elastic.co/t/how-to-understand-registry-file-in-filebeat/157271 "2018-11-21T02:36:02Z")
I know that "registry" is for "tracking files that filebeat is harvesting or is harvested", but for details, how to understand it. Take following as example, what does "timestamp" and "ttl" mean ? { "source": "D:\\aaaa… --- ## [Filebeat 6.4.3 config issue](https://discuss.elastic.co/t/filebeat-6-4-3-config-issue/156366)
**Author:** [@sunny1](https://discuss.elastic.co/u/sunny1)\ **Replies:** 9\ **Last updated:** [November 21, 2018, 12:51am UTC](https://discuss.elastic.co/t/filebeat-6-4-3-config-issue/156366 "2018-11-21T00:51:09Z")
I am installing filebeat 6.4.3. For yml file i have custom index name. When i start filebeat it says setup.template.name and setup.template.patter have to be set if index name is modified. I am not sure how to pass thes… --- ## [Autodiscover AND extract field defined in module](https://discuss.elastic.co/t/autodiscover-and-extract-field-defined-in-module/157422)
**Author:** [@Kaj\_Noppen](https://discuss.elastic.co/u/Kaj_Noppen)\ **Replies:** 2\ **Last updated:** [November 20, 2018, 9:15pm UTC](https://discuss.elastic.co/t/autodiscover-and-extract-field-defined-in-module/157422 "2018-11-20T21:15:23Z")
Hi all, I have been playing around with filebeat on my CentOS machine. I am trying to get filebeat on the host OS to send logs of my Apache)containers to Elasticsearch, whilst also extracting the fields by using the Apa… --- ## [Issue with apache beat module](https://discuss.elastic.co/t/issue-with-apache-beat-module/157601)
**Author:** [@marcandre](https://discuss.elastic.co/u/marcandre)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 6:43pm UTC](https://discuss.elastic.co/t/issue-with-apache-beat-module/157601 "2018-11-20T18:43:49Z")
Hello, I have a lot of apache logs giving me errors: Provided Grok expressions do not match field value: \[- - - \[20/Nov/2018:11:04:32 +0000\] "GET /treasury-services/cash-management/global-cashplus HTTP/1.1" 301 282 "-"… --- ## [Filebeat local send data to logstasho on cloud](https://discuss.elastic.co/t/filebeat-local-send-data-to-logstasho-on-cloud/157416)
**Author:** [@mateo](https://discuss.elastic.co/u/mateo)\ **Replies:** 4\ **Last updated:** [November 20, 2018, 5:49pm UTC](https://discuss.elastic.co/t/filebeat-local-send-data-to-logstasho-on-cloud/157416 "2018-11-20T17:49:14Z")
Good afternoon, someone can tell me how is the process for quer filebeat send information to a logstash that is in the cloud and how to get the information in logstash --- ## [Metricbeat Prometheus module issue with same metric & multiple labels](https://discuss.elastic.co/t/metricbeat-prometheus-module-issue-with-same-metric-multiple-labels/157454)
**Author:** [@gianpietro](https://discuss.elastic.co/u/gianpietro)\ **Replies:** 2\ **Last updated:** [November 20, 2018, 5:40pm UTC](https://discuss.elastic.co/t/metricbeat-prometheus-module-issue-with-same-metric-multiple-labels/157454 "2018-11-20T17:40:16Z")
Hi, I'm using the Prometheus module to grab metrics from an exporter like the following and expose to a Kibana visualization. For example, this one corresponds to "vdu\_name="web01-1-apache\_vdu-1" # HELP osm\_cpu\_utiliza… --- ## [Metricbeat to elasticsearch (same host with docker) connection failed \[forsaken\]](https://discuss.elastic.co/t/metricbeat-to-elasticsearch-same-host-with-docker-connection-failed-forsaken/157525)
**Author:** [@tyteck](https://discuss.elastic.co/u/tyteck)\ **Replies:** 1\ **Last updated:** [November 20, 2018, 4:13pm UTC](https://discuss.elastic.co/t/metricbeat-to-elasticsearch-same-host-with-docker-connection-failed-forsaken/157525 "2018-11-20T16:13:55Z")
Hi there I'm pretty new on docker and elasticsearch and I'm trying to install a supervision for my company. The goal is to have one server hosting both elasticsearch and kibana and many node that are running beats. I … --- ## [Error creating input: Can only start an input when all related states are finished](https://discuss.elastic.co/t/error-creating-input-can-only-start-an-input-when-all-related-states-are-finished/157249)
**Author:** [@Hoon\_Cho](https://discuss.elastic.co/u/Hoon_Cho)\ **Replies:** 3\ **Last updated:** [November 20, 2018, 2:55pm UTC](https://discuss.elastic.co/t/error-creating-input-can-only-start-an-input-when-all-related-states-are-finished/157249 "2018-11-20T14:55:10Z")
I run filebeat with system module enabled and filebeat log like this.. 2018-11-19T07:18:31.662+0900 INFO log/input.go:138 Configured paths: \[/var/log/secure\*\] 2018-11-19T07:18:31.666+0900 ERROR fileset/factory.go:105 … --- ## [Common Practice - Logstash or Directly to Elasticsearch](https://discuss.elastic.co/t/common-practice-logstash-or-directly-to-elasticsearch/157352)
**Author:** [@cmcdowell03](https://discuss.elastic.co/u/cmcdowell03)\ **Replies:** 1\ **Last updated:** [November 20, 2018, 2:21pm UTC](https://discuss.elastic.co/t/common-practice-logstash-or-directly-to-elasticsearch/157352 "2018-11-20T14:21:26Z")
Hello all, Is it common practice to send Metricbeat data and/or Winlogbeat data through Logstash or directly to Elasticsearch? I've noticed in the Kibana UI when I pipe metricbeat through logstash the beats moniotring … --- ## [Regex matching expression to exclude line](https://discuss.elastic.co/t/regex-matching-expression-to-exclude-line/157527)
**Author:** [@Miguel\_Leite](https://discuss.elastic.co/u/Miguel_Leite)\ **Replies:** 2\ **Last updated:** [November 20, 2018, 2:03pm UTC](https://discuss.elastic.co/t/regex-matching-expression-to-exclude-line/157527 "2018-11-20T14:03:40Z")
Hey! I want to exclude a line with filebeat, that is the following: (ERROR) Can't find TpTag 'TagUid' in TpPin Anyone knows how to match that full line with regular expressions? Those apostrophes aren't letting me sta… --- ## [Delaying log harvesting at startup](https://discuss.elastic.co/t/delaying-log-harvesting-at-startup/155513)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 4\ **Last updated:** [November 20, 2018, 1:59pm UTC](https://discuss.elastic.co/t/delaying-log-harvesting-at-startup/155513 "2018-11-20T13:59:00Z")
@pierhugues and @ruflin Since you were who replied to my earlier post . I was setting up Filebeat on a new server, and it seems that if I have external configurations, the reload.period will affect the first harvest ti… --- ## [Filebeat Configuration On excluding files](https://discuss.elastic.co/t/filebeat-configuration-on-excluding-files/157314)
**Author:** [@Miguel\_Leite](https://discuss.elastic.co/u/Miguel_Leite)\ **Replies:** 1\ **Last updated:** [November 20, 2018, 1:55pm UTC](https://discuss.elastic.co/t/filebeat-configuration-on-excluding-files/157314 "2018-11-20T13:55:38Z")
Hello guys! I've been trying to fetch some logs from a specific directory, with enumerous logs files... So I tried the following config: - type: log enabled: true paths: - /base/log/\*.log tags: \["root\_log"\] T… --- ## [Dynamic pipeline selection with Kubernetes Autodiscover hints](https://discuss.elastic.co/t/dynamic-pipeline-selection-with-kubernetes-autodiscover-hints/157444)
**Author:** [@spiffytech](https://discuss.elastic.co/u/spiffytech)\ **Replies:** 1\ **Last updated:** [November 20, 2018, 1:45pm UTC](https://discuss.elastic.co/t/dynamic-pipeline-selection-with-kubernetes-autodiscover-hints/157444 "2018-11-20T13:45:41Z")
I'm using Filebeat in Kubernetes with Autodiscover turned on. How can I route a specific container's logs to a specific ElasticSearch ingest pipeline, so that I can process the logs with the appropriate format grok patte… --- ## [Journalbeat docker image](https://discuss.elastic.co/t/journalbeat-docker-image/157042)
**Author:** [@pastukhov](https://discuss.elastic.co/u/pastukhov)\ **Replies:** 1\ **Last updated:** [November 20, 2018, 1:37pm UTC](https://discuss.elastic.co/t/journalbeat-docker-image/157042 "2018-11-20T13:37:45Z")
Is there any plans to release journalbeat docker image? --- ## [Multiple configuration file with enabled option](https://discuss.elastic.co/t/multiple-configuration-file-with-enabled-option/155899)
**Author:** [@z.fekete](https://discuss.elastic.co/u/z.fekete)\ **Replies:** 10\ **Last updated:** [November 20, 2018, 1:10pm UTC](https://discuss.elastic.co/t/multiple-configuration-file-with-enabled-option/155899 "2018-11-20T13:10:06Z")
I have the following configuration file /etc/filebeat/filebeat.yml: filebeat.config.prospectors: path: /etc/filebeat/conf.d/\*.yml output.logstash: hosts: \["host"\] In each yml file in the folder /etc/filebeat/c… --- ## [How to convert string into integer](https://discuss.elastic.co/t/how-to-convert-string-into-integer/157541)
**Author:** [@godfather7](https://discuss.elastic.co/u/godfather7)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 12:21pm UTC](https://discuss.elastic.co/t/how-to-convert-string-into-integer/157541 "2018-11-20T12:21:45Z")
So i am sending data of event viewer through winlogbeat to kibana. Actually my event\_data.boottime value is coming in string format and i wanted this data field in number format. I have tried changing the format in winlo… [Previous page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=401) [Next page](https://discuss.elastic.co/c/elastic-stack/beats/28.md?page=403)