# Recursive glob patterns syntax help

**URL:** <https://discuss.elastic.co/t/recursive-glob-patterns-syntax-help/117885>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 31, 2018, 7:31pm UTC](https://discuss.elastic.co/t/recursive-glob-patterns-syntax-help/117885 "2018-01-31T19:31:13Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\
**Post date:** [January 31, 2018, 7:31pm UTC](https://discuss.elastic.co/t/recursive-glob-patterns-syntax-help/117885/1 "2018-01-31T19:31:13Z")

</div>

Hello,

I was attempting this pattern

```auto
- input_type: log
  paths:
    - /apps/log/**/catalina.out

  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

  fields_under_root: true
  fields:
    service: my_service

```

To recursively search as the example states: /foo/\*\* expands to /foo, /foo/\*, /foo/\*/\*, and so on, but it did not work for me. I am guessing i have the syntax incorrect. I have used this in the past as apps/log/\*\*/\*.log ... that worked for me, but i only need the catalina.out logs in the example.

Thanks,

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [January 31, 2018, 7:55pm UTC](https://discuss.elastic.co/t/recursive-glob-patterns-syntax-help/117885/2 "2018-01-31T19:55:30Z")

</div>

@cchooks2 Can you start filebeat with the following command:

```auto
./filebeat -c ./filebeat.dev.yml -v -e -d "prospector"

```

You should see how filebeat is expanding your path, I've tested on my side with the following structure:

```auto
ph@sashimi  /tmp/logs  tree
.
└── var
    ├── p
    │ └── c
    │ └── d
    │ ├── dont.log
    │ ├── e
    │ │ ├── dont.log
    │ │ └── findme.log
    │ └── findme.log
    ├── pa
    │ └── d
    │ └── e
    │ └── findme.log
    └── pxx
        └── d

```

With Filebeat configured with the following path `/tmp/logs/var/**/findme.log`, It will find all the files matching this glob pattern.

```auto
2018-01-31T14:52:21.429-0500 DEBUG [prospector] log/prospector.go:361 Check file for harvesting: /tmp/logs/var/p/c/d/findme.log
2018-01-31T14:52:21.429-0500 DEBUG [prospector] log/prospector.go:434 Start harvester for new file: /tmp/logs/var/p/c/d/findme.log
2018-01-31T14:52:21.429-0500 DEBUG [prospector] log/prospector.go:361 Check file for harvesting: /tmp/logs/var/pa/d/e/findme.log
2018-01-31T14:52:21.429-0500 DEBUG [prospector] log/prospector.go:434 Start harvester for new file: /tmp/logs/var/pa/d/e/findme.log
2018-01-31T14:52:21.429-0500 INFO log/harvester.go:216 Harvester started for file: /tmp/logs/var/p/c/d/findme.log
2018-01-31T14:52:21.429-0500 DEBUG [prospector] log/prospector.go:361 Check file for harvesting: /tmp/logs/var/p/c/d/e/findme.log
2018-01-31T14:52:21.429-0500 DEBUG [prospector] log/prospector.go:434 Start harvester for new file: /tmp/logs/var/p/c/d/e/findme.log

```

---

<div class="post-metadata">

**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\
**Post date:** [February 1, 2018, 4:48pm UTC](https://discuss.elastic.co/t/recursive-glob-patterns-syntax-help/117885/3 "2018-02-01T16:48:48Z")

</div>

@pierhugues,

I see the following when i run the command you provided:

```auto
2018/02/01 16:46:27.404657 crawler.go:38: INFO Loading Prospectors: 1
2018/02/01 16:46:27.404667 registrar.go:236: INFO Starting Registrar
2018/02/01 16:46:27.404730 sync.go:41: INFO Start sending events to output
2018/02/01 16:46:27.404786 prospector.go:83: DBG File Configs: [/apps/log/**/catalina.out]
2018/02/01 16:46:27.404798 spooler.go:63: INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2018/02/01 16:46:27.404802 prospector_log.go:44: DBG exclude_files: []
2018/02/01 16:46:27.404999 prospector_log.go:65: INFO Prospector with previous states loaded: 0
2018/02/01 16:46:27.405145 prospector.go:124: INFO Starting prospector of type: log; id: 12907489179318276673
2018/02/01 16:46:27.405175 crawler.go:58: INFO Loading and starting Prospectors completed. Enabled prospectors: 1
2018/02/01 16:46:27.405225 prospector_log.go:70: DBG Start next scan
2018/02/01 16:46:27.408302 prospector_log.go:91: DBG Prospector states cleaned up. Before: 0, After: 0
2018/02/01 16:46:37.408421 prospector.go:183: DBG Run prospector
2018/02/01 16:46:37.408468 prospector_log.go:70: DBG Start next scan
2018/02/01 16:46:37.413157 prospector_log.go:91: DBG Prospector states cleaned up. Before: 0, After: 0
2018/02/01 16:46:47.413346 prospector.go:183: DBG Run prospector
2018/02/01 16:46:47.413372 prospector_log.go:70: DBG Start next scan
2018/02/01 16:46:47.416433 prospector_log.go:91: DBG Prospector states cleaned up. Before: 0, After: 0
2018/02/01 16:46:57.399404 metrics.go:34: INFO No non-zero metrics in the last 30s
2018/02/01 16:46:57.416641 prospector.go:183: DBG Run prospector
2018/02/01 16:46:57.416656 prospector_log.go:70: DBG Start next scan
2018/02/01 16:46:57.419068 prospector_log.go:91: DBG Prospector states cleaned up. Before: 0, After: 0

```

As you can see it is not reading it ... files are actively being written.

Thanks

---

<div class="post-metadata">

**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\
**Post date:** [February 1, 2018, 4:59pm UTC](https://discuss.elastic.co/t/recursive-glob-patterns-syntax-help/117885/4 "2018-02-01T16:59:03Z")

</div>

@pierhugues

When I set it back to /apps/log/\*/\*/catalina.out it picked up all the catalina.out logs. See a snippet below:

```auto
2018/02/01 16:54:44.021731 prospector_log.go:226: DBG Check file for harvesting: /apps/log/aa/bc/catalina.out

2018/02/01 16:54:44.021738 prospector_log.go:259: DBG Update existing file for harvesting: /apps/log/cc/dd/catalina.out, offset: 77138

2018/02/01 16:54:44.021999 prospector_log.go:91: DBG Prospector states cleaned up. Before: 172, After: 172

```

Thanks

---

<div class="post-metadata">

**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\
**Post date:** [February 5, 2018, 6:30pm UTC](https://discuss.elastic.co/t/recursive-glob-patterns-syntax-help/117885/5 "2018-02-05T18:30:42Z")

</div>

The glob is not working as apps/log/\*\*/catalina.out, but is as apps/log/\*/\*/catalina.out.

Any thoughts as to why? The inputs are above.

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 5, 2018, 6:30pm UTC](https://discuss.elastic.co/t/recursive-glob-patterns-syntax-help/117885/6 "2018-03-05T18:30:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.