# Unable to harvest log files with glob

**URL:** <https://discuss.elastic.co/t/unable-to-harvest-log-files-with-glob/253849>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 30, 2020, 5:51pm UTC](https://discuss.elastic.co/t/unable-to-harvest-log-files-with-glob/253849 "2020-10-30T17:51:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arvind\_Ks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arvind_ks/32/77463_2.png) [@Arvind\_Ks](https://discuss.elastic.co/u/Arvind_Ks)\
**Post date:** [October 30, 2020, 5:51pm UTC](https://discuss.elastic.co/t/unable-to-harvest-log-files-with-glob/253849/1 "2020-10-30T17:51:31Z")

</div>

Hi Team,  
I am trying to parse jenkins build logs using filebeat. There are several subdirectories that it needs to traverse to. I can also see in the filebeat log what all locations filebeat is checking. However when i create a new log file or add entry to existing log files, logstash is not harvesting those changes. Please help. If i use just a single path instead of 3 file path in my case, it seems to work.

My Filebeat config

```auto
[ashivaramakrishnan@ci001.atl1.turn.com filebeat]$ more conf.d/jenkins-ci.yml    
filebeat:
  inputs:
    - paths:
      - "/var/lib/jenkins/jobs/cd-pipeline-master*/builds/*/junitResult.xml"
      - "/var/lib/jenkins/jobs/cd-pipeline-release*/builds/*/junitResult.xml"
      - "/var/lib/jenkins/jobs/cd-pipeline-production*/builds/**/junitResult.xml"
      encoding: plain
      fields:
        timezone: PDT
        log_type: jenkins_junit
      close_renamed: true
      close_removed: true
      fields_under_root: true
      type: log
      ignore_older: 5m
      scan_frequency: 10s
      harvester_buffer_size: 16384

```

Filebeat log files::

```auto
Configured paths: [
/var/lib/jenkins/jobs/cd-pipeline-master*/builds/*/junitResult.xml 
/var/lib/jenkins/jobs/cd-pipeline-release*/builds/*/junitResult.xml 
/var/lib/jenkins/jobs/cd-pipeline-production*/builds/junitResult.xml 
/var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/*/*/*/junitResult.xml /var/lib/jenkins/jobs/cd-pipeline-production*/builds/*/*/*/*/*/*/*/*/junitResult.xml]

```

For example if try modifying a file in the below path. filebeat doesnt harvest the file.  
eg::

```auto
/var/lib/jenkins/jobs/cd-pipeline-master-ucheetahclient-test/builds/0000/junitResult.xml

```

Thanks  
Arvind

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [November 4, 2020, 3:09pm UTC](https://discuss.elastic.co/t/unable-to-harvest-log-files-with-glob/253849/2 "2020-11-04T15:09:24Z")

</div>

Hi @Arvind_Ks 🙂 I'm not fully sure if this is supported but I think that when using folders you need to use `**` (double asterisk). So `cd-pipeline-master*` must be `cd-pipeline-master**` for example. Can you give it a try?

---

<div class="post-metadata">

**Author:** ![Arvind\_Ks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arvind_ks/32/77463_2.png) [@Arvind\_Ks](https://discuss.elastic.co/u/Arvind_Ks)\
**Post date:** [November 4, 2020, 3:28pm UTC](https://discuss.elastic.co/t/unable-to-harvest-log-files-with-glob/253849/3 "2020-11-04T15:28:10Z")

</div>

Hi Mario,  
Thanks for replying back. Actually when i used "\*\*" it was searching upto 8 sub-directories and filebeat was running slowly. Now i replaced that with a single "\*" and it appears to be working fast.

Thanks  
Arvind

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 2, 2020, 3:28pm UTC](https://discuss.elastic.co/t/unable-to-harvest-log-files-with-glob/253849/4 "2020-12-02T15:28:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.