# Filebeat not considering all file matched via wildcard (\*) **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [July 26, 2016, 2:15pm UTC](https://discuss.elastic.co/t/filebeat-not-considering-all-file-matched-via-wildcard/56406 "2016-07-26T14:15:15Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![s8sg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s8sg/32/12274_2.png) [@s8sg](https://discuss.elastic.co/u/s8sg)\ **Post date:** [July 26, 2016, 2:15pm UTC](https://discuss.elastic.co/t/filebeat-not-considering-all-file-matched-via-wildcard/56406/1 "2016-07-26T14:15:15Z")
I'm using **filebeat** from an custom **docker container**. Where **/var/log** is mounted as **/data/system** and **/var/lib/nomad/alloc** as **/data/nomad** I have the below files in the mounted directory: **1. /data/system** `$ docker exec 4742ce2ddd47 ls -la /data/system` ```auto drwxr-xr-x 2 root root 6 Feb 22 17:21 anaconda drwxr-x--- 2 root root 94 Jul 26 02:31 audit -rw-r--r-- 1 root root 935 Jul 25 06:00 boot.log -rw------- 1 root sshd 3091968 Jul 26 14:09 btmp -rw------- 1 root sshd 48384 Jul 1 02:52 btmp-20160701 drwxrwx--- 2 167 167 16384 Jul 26 10:29 ceph drwxr-xr-x 2 996 993 6 Nov 23 2015 chrony -rw-r--r-- 1 root root 16003 Jul 25 06:00 cloud-init-output.log -rw-r--r-- 1 root root 24078 Jul 25 06:00 cloud-init.log -rw-r--r-- 1 root root 17676 Jul 26 14:01 cron -rw-r--r-- 1 root root 35633 Jul 3 03:42 cron-20160703 -rw-r--r-- 1 root root 97698 Jul 11 03:48 cron-20160711 -rw-r--r-- 1 root root 47765 Jul 15 01:01 cron-20160717 -rw-r--r-- 1 root root 68905 Jul 25 03:38 cron-20160725 -rw-r--r-- 1 root root 36300 Jul 25 06:00 dmesg -rw-r--r-- 1 root root 36141 Jul 19 07:28 dmesg.old -rw-r--r-- 1 root root 292292 Jul 26 14:23 lastlog -rw------- 1 root root 1641 Jul 26 03:41 maillog -rw------- 1 root root 2364 Jul 2 03:46 maillog-20160703 -rw------- 1 root root 6098 Jul 10 03:22 maillog-20160711 -rw------- 1 root root 3064 Jul 14 03:28 maillog-20160717 -rw------- 1 root root 3780 Jul 24 03:12 maillog-20160725 -rw------- 1 root root 207185182 Jul 26 14:34 messages -rw------- 1 root root 11069008 Jul 3 03:41 messages-20160703 -rw------- 1 root root 93373525 Jul 11 03:47 messages-20160711 -rw------- 1 root root 93607378 Jul 15 01:17 messages-20160717 -rw------- 1 root root 1251294825 Jul 25 03:38 messages-20160725 drwxr-xr-x 2 38 38 6 May 31 10:29 ntpstats drwx------ 2 root root 6 Jun 10 2014 ppp -rw------- 1 root root 676596 Jul 26 14:26 secure -rw------- 1 root root 340811 Jul 3 03:41 secure-20160703 -rw------- 1 root root 1834918 Jul 11 03:47 secure-20160711 -rw------- 1 root root 1955397 Jul 15 01:17 secure-20160717 -rw------- 1 root root 1242832 Jul 25 03:36 secure-20160725 -rw------- 1 root root 0 Jul 25 03:38 spooler -rw------- 1 root root 0 Feb 22 17:12 spooler-20160703 -rw------- 1 root root 0 Jul 3 03:42 spooler-20160711 -rw------- 1 root root 0 Jul 11 03:48 spooler-20160717 -rw------- 1 root root 0 Jul 17 03:21 spooler-20160725 -rw------- 1 root root 0 Feb 22 17:10 tallylog drwxr-xr-x 2 root root 22 Jun 30 07:18 tuned -rw-rw-r-- 1 root sshd 640512 Jul 26 14:26 wtmp -rw-rw-r-- 1 root sshd 1325952 Jul 6 12:03 wtmp-20160707 -rw------- 1 root root 6248 Jul 26 04:58 yum.log ``` **2. /data/nomad** `$ docker exec 4742ce2ddd47 ls -la /data/nomad` ```auto drwxr-xr-x 4 root root 43 Jul 26 09:43 33af0f4c-55a9-41f0-70d4-5152ea4d4bbf drwxr-xr-x 4 root root 47 Jul 26 10:29 63356737-070f-c22c-3c03-0a7c1aae17d1 ``` `$ docker exec 4742ce2ddd47 ls /data/nomad/33af0f4c../alloc/logs` ```auto gslb-enduser-agent.stderr.0 gslb-enduser-agent.stdout.0 ``` **My Filebeat Configuration is defined as:** `$ docker exec 4742ce2ddd47 cat filebeat/filebeat.yml` ```auto filebeat: # List of prospectors to fetch data. prospectors: - paths: ["/data/system/*", "/data/nomad/*/alloc/logs/*"] input_type: log output: elasticsearch: enabled: true hosts: ["172.31.9.202:9200", "172.31.4.4:9200", "172.31.2.180:9200"] index: "filebeat" template: name: "filebeat" path: "/filebeat/filebeat.template.json" overwrite: true # LogStash output is not used # There are three options for the log ouput: syslog, file, stderr. logging: to_syslog: false to_files: false ``` **But in kibana I'm getting only below files in the table** ```auto /data/system/messages-20160703 1,944 /data/system/messages 1,403 /data/system/secure-20160725 1,153 /data/nomad/33af0f4c-55a9-41f0-70d4-5152ea4d4bbf/alloc/logs/gslb-enduser-agent.stderr.0 968 /data/system/secure-20160717 880 /data/system/messages-20160725 699 /data/system/messages-20160711 238 /data/system/messages-20160717 156 /data/system/secure-20160711 150 /data/system/secure 72 ``` Is there is anything wrong I'm doing ? **Please Help**. Thanks in advance 🙂 ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [July 26, 2016, 9:55pm UTC](https://discuss.elastic.co/t/filebeat-not-considering-all-file-matched-via-wildcard/56406/2 "2016-07-26T21:55:49Z")
I think the issue is with the double wildcard used in the path. See [https://github.com/elastic/beats/issues/2084](https://github.com/elastic/beats/issues/2084) ---
**Author:** ![s8sg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s8sg/32/12274_2.png) [@s8sg](https://discuss.elastic.co/u/s8sg)\ **Post date:** [July 28, 2016, 12:47am UTC](https://discuss.elastic.co/t/filebeat-not-considering-all-file-matched-via-wildcard/56406/3 "2016-07-28T00:47:26Z")
Hi, I have tested the same file path patterns manually by `filepath.Glob()` inside the container. It is considering all the files. Is there is any other setting that I'm doing wrong? Thanks ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [July 31, 2016, 7:44pm UTC](https://discuss.elastic.co/t/filebeat-not-considering-all-file-matched-via-wildcard/56406/4 "2016-07-31T19:44:49Z")
What do you mean by "tested manually"? Did you write a small Golang App? ---
**Author:** ![s8sg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s8sg/32/12274_2.png) [@s8sg](https://discuss.elastic.co/u/s8sg)\ **Post date:** [August 5, 2016, 9:19am UTC](https://discuss.elastic.co/t/filebeat-not-considering-all-file-matched-via-wildcard/56406/5 "2016-08-05T09:19:08Z")
Yes. But it was not an issue. I was using kibana and it was only getting me 20 result as per the settings. Thanks for the support 🙂 ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [August 8, 2016, 8:34am UTC](https://discuss.elastic.co/t/filebeat-not-considering-all-file-matched-via-wildcard/56406/6 "2016-08-08T08:34:29Z")
@s8sg Nice. Any chance to share the code you wrote to compare it and do some additional testing. Perhaps you can directly chime in on [https://github.com/elastic/beats/issues/2084](https://github.com/elastic/beats/issues/2084) and add the details of your testing there. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 16, 2016, 2:15pm UTC](https://discuss.elastic.co/t/filebeat-not-considering-all-file-matched-via-wildcard/56406/7 "2016-08-16T14:15:14Z")
This topic was automatically closed after 21 days. New replies are no longer allowed.