# How to enable filebeat to harvest .File Type? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [July 6, 2018, 10:15pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986 "2018-07-06T22:15:30Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![krisdinesh](https://avatars.discourse-cdn.com/v4/letter/k/f14d63/32.png) [@krisdinesh](https://discuss.elastic.co/u/krisdinesh)\ **Post date:** [July 6, 2018, 10:15pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/1 "2018-07-06T22:15:30Z")
Hello, Please help me here. I have a file needs to be harvested by Filebeats 'D:\apps\jenkins\jobs\TestJob\builds\180\log.File' but its not. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [July 8, 2018, 1:08pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/2 "2018-07-08T13:08:32Z")
This should work: ```auto filebeat.inputs: - type: log paths: - 'D:\apps\jenkins\jobs\TestJob\builds\180\log.File' ``` But if you are using a glob then you might be running into this issue: [https://github.com/elastic/beats/issues/6737](https://github.com/elastic/beats/issues/6737) ---
**Author:** ![krisdinesh](https://avatars.discourse-cdn.com/v4/letter/k/f14d63/32.png) [@krisdinesh](https://discuss.elastic.co/u/krisdinesh)\ **Post date:** [July 8, 2018, 3:32pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/3 "2018-07-08T15:32:25Z")
I appreciate your response in this regard. I tried exactly as you told before raising this ticket where filebeat was not harvesting the file. I just retried one more time to retry as you suggested and looks like still filebeat is still not harvesting the file. Please note that I cleared off the registry files every time before starting the filebeat so that it harvest files though it is already processed. But instead of file path 'D:\apps\jenkins\jobs\SWX-Regression\builds\180\log.File’ if I give the path like this its harvesting 'D:\apps\jenkins\jobs\SWX-Regression\builds\180\log’. \*\*Below is my .yml \*\* ```auto #=========================== Filebeat inputs ============================= filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. - type: log # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - 'D:\apps\jenkins\jobs\SWX-Regression\builds\180\log.File' ``` **Filebeat Logs:** ```auto 2018-07-08T10:04:15.887-0500 INFO instance/beat.go:492 Home path: [D:\apps\filebeat] Config path: [D:\apps\filebeat] Data path: [C:\ProgramData\filebeat] Logs path: [C:\ProgramData\filebeat\logs] 2018-07-08T10:04:15.891-0500 INFO instance/beat.go:499 Beat UUID: 6cba9929-4375-40d7-ac7e-7403146e2595 2018-07-08T10:04:15.892-0500 INFO [beat] instance/beat.go:716 Beat info {"system_info": {"beat": {"path": {"config": "D:\\apps\\filebeat", "data": "C:\\ProgramData\\filebeat", "home": "D:\\apps\\filebeat", "logs": "C:\\ProgramData\\filebeat\\logs"}, "type": "filebeat", "uuid": "6cba9929-4375-40d7-ac7e-7403146e2595"}}} 2018-07-08T10:04:15.892-0500 INFO [beat] instance/beat.go:725 Build info {"system_info": {"build": {"commit": "a04cb664d5fbd4b1aab485d1766f3979c138fd38", "libbeat": "6.3.0", "time": "2018-06-11T22:34:03.000Z", "version": "6.3.0"}}} 2018-07-08T10:04:15.892-0500 INFO [beat] instance/beat.go:728 Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":8,"version":"go1.9.4"}}} 2018-07-08T10:04:15.899-0500 INFO [beat] instance/beat.go:732 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-07-04T17:56:51.81-05:00","hostname":"dwvmlqaci01","ips":["fe80::128:c796:faf8:3236/64","10.32.44.57/23","::1/128","127.0.0.1/8"],"kernel_version":"6.3.9600.19038 (winblue_ltsb_escrow.180608-1416)","mac_addresses":["00:50:56:85:53:13","00:00:00:00:00:00:00:e0"],"os":{"family":"windows","platform":"windows","name":"Windows Server 2012 R2 Standard","version":"6.3","major":3,"minor":0,"patch":0,"build":"9600.19038"},"timezone":"CDT","timezone_offset_sec":-18000,"id":"4a17570a-880e-4fff-b735-9c69f30d9930"}}} 2018-07-08T10:04:15.899-0500 INFO instance/beat.go:225 Setup Beat: filebeat; Version: 6.3.0 2018-07-08T10:04:15.900-0500 INFO pipeline/module.go:81 Beat name: dwvmlqaci01 2018-07-08T10:04:15.902-0500 INFO instance/beat.go:315 filebeat start running. 2018-07-08T10:04:15.902-0500 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s 2018-07-08T10:04:15.902-0500 INFO registrar/registrar.go:75 No registry file found under: C:\ProgramData\filebeat\registry. Creating a new registry file. 2018-07-08T10:04:15.943-0500 INFO registrar/registrar.go:112 Loading registrar data from C:\ProgramData\filebeat\registry 2018-07-08T10:04:15.943-0500 INFO registrar/registrar.go:123 States Loaded from registrar: 0 2018-07-08T10:04:15.943-0500 WARN beater/filebeat.go:354 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning. 2018-07-08T10:04:15.943-0500 INFO crawler/crawler.go:48 Loading Inputs: 1 2018-07-08T10:04:15.944-0500 INFO log/input.go:111 Configured paths: [D:\apps\jenkins\jobs\SWX-Regression\builds\180\log.File] 2018-07-08T10:04:15.944-0500 INFO input/input.go:87 Starting input of type: log; ID: 4988709827331553149 2018-07-08T10:04:15.944-0500 INFO crawler/crawler.go:82 Loading and starting Inputs completed. Enabled inputs: 1 2018-07-08T10:04:15.944-0500 INFO cfgfile/reload.go:122 Config reloader started 2018-07-08T10:04:45.990-0500 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":125,"time":{"ms":125}},"total":{"ticks":171,"time":{"ms":171},"value":171},"user":{"ticks":46,"time":{"ms":46}}},"info":{"ephemeral_id":"ba8f2ab1-ac14-4d1d-8ca6-b0bc9c314c72","uptime":{"ms":30049}},"memstats":{"gc_next":4194304,"memory_alloc":1592600,"memory_total":3123176,"rss":16773120}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":3},"output":{"type":"logstash"},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0},"writes":1},"system":{"cpu":{"cores":8}}}}} 2018-07-08T10:05:15.906-0500 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":125},"total":{"ticks":171,"value":171},"user":{"ticks":46}},"info":{"ephemeral_id":"ba8f2ab1-ac14-4d1d-8ca6-b0bc9c314c72","uptime":{"ms":60051}},"memstats":{"gc_next”:4194 ``` Thanks Dinesh ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [July 9, 2018, 6:38pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/4 "2018-07-09T18:38:28Z")
> [@krisdinesh](#): > > But instead of file path 'D:\apps\jenkins\jobs\SWX-Regression\builds\180\log.File’ if I give the path like this its harvesting 'D:\apps\jenkins\jobs\SWX-Regression\builds\180\log’. Can you please post a directory listing from the cli too? Like `PS > dir D:\apps\jenkins\jobs\SWX-Regression\builds\180` And I think it would be useful to see the debug logs from Filebeat as it starts up and initializes the inputs and harvesters. If you add `logging.level: debug` to your config this will enable debug. ---
**Author:** ![krisdinesh](https://avatars.discourse-cdn.com/v4/letter/k/f14d63/32.png) [@krisdinesh](https://discuss.elastic.co/u/krisdinesh)\ **Post date:** [July 9, 2018, 9:57pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/5 "2018-07-09T21:57:30Z")
Hi Andrew, Here is the snap of the dir command for the particular directory and also file explorer snap as well ![](https://us1.discourse-cdn.com/elastic/original/3X/7/d/7dfd4db89db901a1a9429c1e6f8e29a4cfe5121e.png) ![](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3ce2ecb0d7f5660ac61e945c70c522b77c04bb40.jpg) ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [July 10, 2018, 9:35pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/6 "2018-07-10T21:35:55Z")
Based on the `dir` output, the actual file is called `log` and does not have a file extension so this explains why `'D:\apps\jenkins\jobs\SWX-Regression\builds\180\log'` works. ---
**Author:** ![krisdinesh](https://avatars.discourse-cdn.com/v4/letter/k/f14d63/32.png) [@krisdinesh](https://discuss.elastic.co/u/krisdinesh)\ **Post date:** [July 10, 2018, 9:57pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/7 "2018-07-10T21:57:40Z")
Thanks Andrew for the clarification. I am trying to parse through filbeat as the path configured in .yml as 'D:\apps\jenkins\jobs\SWX-Regression\builds\*\*\log’ but is there any way to parse only the folders that contains only number? ![](https://us1.discourse-cdn.com/elastic/original/3X/8/a/8a45b7d4e073e2fd80604a269a8872951cdf4d07.png) Regards, Dinesh ---
**Author:** ![Wahyu](https://avatars.discourse-cdn.com/v4/letter/w/8491ac/32.png) [@Wahyu](https://discuss.elastic.co/u/Wahyu)\ **Post date:** [July 12, 2018, 2:39am UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/8 "2018-07-12T02:39:17Z")
@krisdinesh After you succeeded to run the filebeat, do you use logstash and kibana to catch the harvest of filebeat ? ---
**Author:** ![krisdinesh](https://avatars.discourse-cdn.com/v4/letter/k/f14d63/32.png) [@krisdinesh](https://discuss.elastic.co/u/krisdinesh)\ **Post date:** [July 12, 2018, 2:51am UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/9 "2018-07-12T02:51:23Z")
Yes Putra ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [July 12, 2018, 11:32am UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/10 "2018-07-12T11:32:40Z")
> [@krisdinesh](#): > > I am trying to parse through filbeat as the path configured in .yml as 'D:\apps\jenkins\jobs\SWX-Regression\builds\*\*\log’ but is there any way to parse only the folders that contains only number? The glob patterns that are supported are described in detail in the [filepath.Match](https://golang.org/pkg/path/filepath/#Match) docs from Go. In addition to those patterns we added support for `**` to enable recursive matching. So `D:\apps\jenkins\jobs\SWX-Regression\builds\*\log` is probably what you want to use. Try that. ---
**Author:** ![krisdinesh](https://avatars.discourse-cdn.com/v4/letter/k/f14d63/32.png) [@krisdinesh](https://discuss.elastic.co/u/krisdinesh)\ **Post date:** [July 13, 2018, 11:46pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/11 "2018-07-13T23:46:15Z")
Thanks Andrew!! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 10, 2018, 11:46pm UTC](https://discuss.elastic.co/t/how-to-enable-filebeat-to-harvest-file-type/138986/12 "2018-08-10T23:46:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.