# How can I send custom logs in a specific location to filebeat running inside docker

**URL:** <https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [December 28, 2022, 3:11pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106 "2022-12-28T15:11:57Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 28, 2022, 3:11pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/1 "2022-12-28T15:11:57Z")

</div>

I am new to filebeat and elk. I am trying to send custom logs using filebeat to Elasticsearch directly.Both the elk stack and filebeat are running inside docker containers.. The custom logs are in the folder home/username/docker/hello.log. Here is my filebeat.yml file:

```auto
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/raju/elk/docker/*.log
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

processors:
- add_cloud_metadata: ~

output.elasticsearch:
  hosts: ["my_ip:9200"]

```

And here is my custom log file:

```auto
This is a custom log file 
Sending logs to elastic search

```

And these are the commands using which I am using to run filebeat:

```auto
docker run -d \
  --name=filebeat \
  --user=root \
  --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  docker.elastic.co/beats/filebeat:8.5.3 filebeat -e --strict.perms=false

```

When i use the above commands to run filebeat I can see the logs of the docker containers on my kibana dashboard. But I am struggling on how to make filebeat to read my custom logs from the specified location above and show me the lines inside the log file.

Anyhelp would be appreciated.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2022, 3:00pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/2 "2022-12-29T15:00:34Z")

</div>

> [@Raja\_Muneer](#):
>
> home/username/docker/hello.log

If those files are on the host system you will need to mount those to the filebeat container so the filebeat container can access them

By default a docker container is isolated from the host filesystem.

Perhaps you should read about docker accessing host filesystems.[here](https://docs.docker.com/storage/)

This mounted the filebeat.yml

`--volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro`

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 3:13pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/3 "2022-12-29T15:13:36Z")

</div>

@stephenb Thank you for valuable reply. I did mount the current directory as a volume to docker container running filebeat but still I can't access the log file and kibana shows no index pattern. Maybe I am not sure if log file needs a certain format or not.  
--volume="$(pwd)/hello.log:/usr/share/filebeat/hello.log:ro

I guess you mean like this. Path inside docker can be changed I am sure but it has no effect as it is not reading the lines inside my log file

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2022, 3:22pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/4 "2022-12-29T15:22:55Z")

</div>

This is more of a docker issue than a filebest issue.

I would mount the file system / directory and then exec into the filebeat docker container and see if you can see those files.... Once you can see those files ... set up filebeat correctly.

Until file beat can access those files, it clearly won't be able to ship them to Kibana.

Perhaps you should try filebeat outside of docker first. Get that working correct.... And then try it with docker.

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 3:30pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/5 "2022-12-29T15:30:56Z")

</div>

```auto
2022-12-29T15:37:35.532Z	INFO	instance/beat.go:645	Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2022-12-29T15:37:35.672Z	INFO	instance/beat.go:653	Beat ID: cf45bdc7-e611-4f0d-a384-89d6853908ab
2022-12-29T15:37:35.673Z	INFO	[seccomp]	seccomp/seccomp.go:124	Syscall filter successfully installed
2022-12-29T15:37:35.673Z	INFO	[beat]	instance/beat.go:981	Beat info	{"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "cf45bdc7-e611-4f0d-a384-89d6853908ab"}}}
2022-12-29T15:37:35.674Z	INFO	[beat]	instance/beat.go:990	Build info	{"system_info": {"build": {"commit": "1da173a9e716715a7a54bb3ff4db05b5c24fc8ce", "libbeat": "7.10.1", "time": "2020-12-04T23:27:17.000Z", "version": "7.10.1"}}}
2022-12-29T15:37:35.674Z	INFO	[beat]	instance/beat.go:993	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.14.12"}}}
2022-12-29T15:37:35.675Z	INFO	[beat]	instance/beat.go:997	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2022-12-29T08:33:31Z","containerized":false,"name":"0c5e284a3f33","ip":["127.0.0.1/8","172.17.0.2/16"],"kernel_version":"5.15.0-56-generic","mac":["02:42:ac:11:00:02"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":9,"patch":2009,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0,"id":"d624e9f79ced4be589fadca86f6cdcf0"}}}
2022-12-29T15:37:35.708Z	INFO	[beat]	instance/beat.go:1026	Process info	{"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 1, "ppid": 0, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2022-12-29T15:37:34.190Z"}}}
2022-12-29T15:37:35.708Z	INFO	instance/beat.go:299	Setup Beat: filebeat; Version: 7.10.1
2022-12-29T15:37:35.708Z	INFO	[index-management]	idxmgmt/std.go:184	Set output.elasticsearch.index to 'filebeat-7.10.1' as ILM is enabled.
2022-12-29T15:37:35.709Z	INFO	eslegclient/connection.go:99	elasticsearch url: http://192.168.1.38:9200
2022-12-29T15:37:35.709Z	INFO	[publisher]	pipeline/module.go:113	Beat name: 0c5e284a3f33
2022-12-29T15:37:35.710Z	INFO	[monitoring]	log/log.go:118	Starting metrics logging every 30s
2022-12-29T15:37:35.710Z	INFO	instance/beat.go:455	filebeat start running.
2022-12-29T15:37:35.842Z	INFO	memlog/store.go:119	Loading data file of '/usr/share/filebeat/data/registry/filebeat' succeeded. Active transaction id=0
2022-12-29T15:37:35.842Z	INFO	memlog/store.go:124	Finished loading transaction log file for '/usr/share/filebeat/data/registry/filebeat'. Active transaction id=0
2022-12-29T15:37:35.843Z	INFO	[registrar]	registrar/registrar.go:109	States Loaded from registrar: 0
2022-12-29T15:37:35.843Z	INFO	[crawler]	beater/crawler.go:71	Loading Inputs: 1
2022-12-29T15:37:35.843Z	INFO	log/input.go:157	Configured paths: [/home/raju/elk/docker/hello.log]
2022-12-29T15:37:35.843Z	INFO	[crawler]	beater/crawler.go:141	Starting input (ID: 273078644940147100)
2022-12-29T15:37:35.844Z	INFO	[crawler]	beater/crawler.go:108	Loading and starting Inputs completed. Enabled inputs: 1
2022-12-29T15:37:35.844Z	INFO	cfgfile/reload.go:164	Config reloader started
2022-12-29T15:37:35.844Z	INFO	cfgfile/reload.go:224	Loading of config files completed.
2022-12-29T15:37:38.708Z	INFO	[add_cloud_metadata]	add_cloud_metadata/add_cloud_metadata.go:89	add_cloud_metadata: hosting provider type not detected.
2022-12-29T15:38:05.714Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":32}},"total":{"ticks":180,"time":{"ms":187},"value":180},"user":{"ticks":150,"time":{"ms":155}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"de713df3-878c-4a42-9d81-5ae685da0552","uptime":{"ms":30234}},"memstats":{"gc_next":17209040,"memory_alloc":9424872,"memory_total":44276904,"rss":69820416},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"type":"elasticsearch"},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":4},"load":{"1":0.59,"15":0.81,"5":0.82,"norm":{"1":0.1475,"15":0.2025,"5":0.205}}}}}}
2022-12-29T15:38:35.729Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":1}},"total":{"ticks":190,"time":{"ms":6},"value":190},"user":{"ticks":160,"time":{"ms":5}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"de713df3-878c-4a42-9d81-5ae685da0552","uptime":{"ms":60249}},"memstats":{"gc_next":17209040,"memory_alloc":9662192,"memory_total":44514224,"rss":-1777664},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.68,"15":0.82,"5":0.82,"norm":{"1":0.17,"15":0.205,"5":0.205}}}}}}
2022-12-29T15:39:05.716Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":{"ms":9}},"total":{"ticks":200,"time":{"ms":11},"value":200},"user":{"ticks":160,"time":{"ms":2}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"de713df3-878c-4a42-9d81-5ae685da0552","uptime":{"ms":90235}},"memstats":{"gc_next":17209040,"memory_alloc":10084008,"memory_total":44936040,"rss":-950272},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.58,"15":0.81,"5":0.79,"norm":{"1":0.145,"15":0.2025,"5":0.1975}}}}}}
2022-12-29T15:39:35.781Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":{"ms":5}},"total":{"ticks":220,"time":{"ms":29},"value":220},"user":{"ticks":180,"time":{"ms":24}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"de713df3-878c-4a42-9d81-5ae685da0552","uptime":{"ms":120233}},"memstats":{"gc_next":17836992,"memory_alloc":10144576,"memory_total":45168992},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.35,"15":0.78,"5":0.71,"norm":{"1":0.0875,"15":0.195,"5":0.1775}}}}}}
2022-12-29T15:40:05.715Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":80,"time":{"ms":37}},"total":{"ticks":300,"time":{"ms":80},"value":300},"user":{"ticks":220,"time":{"ms":43}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"de713df3-878c-4a42-9d81-5ae685da0552","uptime":{"ms":150237}},"memstats":{"gc_next":17836992,"memory_alloc":9252552,"memory_total":45503528},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.28,"15":0.76,"5":0.66,"norm":{"1":0.07,"15":0.19,"5":0.165}}}}}}
2022-12-29T15:40:35.715Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":80,"time":{"ms":3}},"total":{"ticks":310,"time":{"ms":9},"value":310},"user":{"ticks":230,"time":{"ms":6}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"de713df3-878c-4a42-9d81-5ae685da0552","uptime":{"ms":180233}},"memstats":{"gc_next":17836992,"memory_alloc":9590776,"memory_total":45841752},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.71,"15":0.77,"5":0.71,"norm":{"1":0.1775,"15":0.1925,"5":0.1775}}}}}}
2022-12-29T15:41:05.715Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":90,"time":{"ms":5}},"total":{"ticks":330,"time":{"ms":11},"value":330},"user":{"ticks":240,"time":{"ms":6}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"de713df3-878c-4a42-9d81-5ae685da0552","uptime":{"ms":210233}},"memstats":{"gc_next":17836992,"memory_alloc":9917784,"memory_total":46168760},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.43,"15":0.75,"5":0.64,"norm":{"1":0.1075,"15":0.1875,"5":0.16}}}}}}
2022-12-29T15:41:35.712Z	INFO	[monitoring]	log/log.go:145	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":90,"time":{"ms":1}},"total":{"ticks":330,"time":{"ms":8},"value":330},"user":{"ticks":240,"time":{"ms":7}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"de713df3-878c-4a42-9d81-5ae685da0552","uptime":{"ms":240234}},"memstats":{"gc_next":17836992,"memory_alloc":10332720,"memory_total":46583696},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.46,"15":0.74,"5":0.63,"norm":{"1":0.115,"15":0.185,"5":0.1575}}}}}}

```

These are the logs from filebeat I guess it is reading the file.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2022, 3:35pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/6 "2022-12-29T15:35:20Z")

</div>

There should be some more logs about how many it published, shipped and acked....

Also, it'll only read the file once so it already read it once It's not going to read it again unless you destroy the container and recreate it because it keeps track of what it's read.

See those long lines about the registry that's keeping track of what it already read. So running filebeat over and over on the same log file does not work. That's a fundamental concept of filebeat unless you clean out the registry or more log lines or added to your file.

You could try a appending some more lines to your log file and see what happens.

Again, if you're just beginning I would probably run file beat first, not in docker. You're actually trying to make it do something that's a little more advanced.

Also, why such an old version? 7.10

You should be running 7.17 at least

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 3:50pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/7 "2022-12-29T15:50:31Z")

</div>

I have added the full output of filebeat by editing my previous reply. I can see the mount is successful inside docker.

```auto
sh-4.2# ls
LICENSE.txt NOTICE.txt README.md data fields.yml filebeat	filebeat.reference.yml	filebeat.yml kibana logs logstash.log logstash-json.log~ module modules.d
sh-4.2# 

```

Here logstash.log is the log file I added. It was working with this command:

```auto
docker run -d \
  --name=filebeat \
  --user=root \
  --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  docker.elastic.co/beats/filebeat:8.5.3 filebeat -e --strict.perms=false

```

As it is looking for running containers on my system and is logging everything to my kibana dashboard. But I am not sure why its not working with a custom log file. Maybe you are right I need to test it by installing filebeat on my local system.

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 3:54pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/8 "2022-12-29T15:54:03Z")

</div>

I appreciate your patience. But I am not sure maybe I need to give the path of log file inside docker after mounting in filebeat.yml instead of giving localhost path like home/raju/elk/docker/.log to path inside docker /usr/share/filbeat/.log. I am not sure maybe that it would pick up.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2022, 3:58pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/9 "2022-12-29T15:58:37Z")

</div>

Yes, it has to be path in the docker container

I'm trying to give you suggestions. I suggested that you exec into the docker container once it is running and look for the path for your log file. That is the path that has to be in the yml file that filebeat uses

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2022, 4:01pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/10 "2022-12-29T16:01:25Z")

</div>

> [@Raja\_Muneer](#):
>
> ```auto
> paths:
> - /home/raju/elk/docker/*.log
> 
> ```

That should be the path in the docker container

Again, this is an understanding about how docker works Perhaps spend a little more time understanding the interaction between docker and the host

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 4:02pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/11 "2022-12-29T16:02:54Z")

</div>

I did show the output of docker container above by exec into the container. I see both the filebeat.yml and log file I added from localhost are there and have some content that I specified from my localhost.

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 4:04pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/12 "2022-12-29T16:04:55Z")

</div>

Here is the output of the filebeat.yml inside docker container.

```auto
sh-4.2# ls
LICENSE.txt NOTICE.txt README.md data fields.yml filebeat	filebeat.reference.yml	filebeat.yml kibana logs logstash-json.log module modules.d
sh-4.2# cat filebeat.yml
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false
filebeat.inputs:
- type: log
  paths:
    - /home/raju/elk/docker/hello.log
#filebeat.autodiscover:
 # providers:
  # - type: docker
   # hints.enabled: true

processors:
- add_cloud_metadata: ~

output.elasticsearch:
  hosts: '${ELASTICSEARCH_HOSTS:192.168.1.38:9200}'
# username: '${ELASTICSEARCH_USERNAME:}'
# password: '${ELASTICSEARCH_PASSWORD:}'
sh-4.2#    

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2022, 4:06pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/13 "2022-12-29T16:06:16Z")

</div>

> [@Raja\_Muneer](#):
>
> ```auto
> paths:
> - /home/raju/elk/docker/hello.log
> 
> ```

Shouldn't that be the path inside the docker container ....not the host path

What does your mount look for that Please show

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 4:11pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/14 "2022-12-29T16:11:33Z")

</div>

Here is how I am mounting the file. Please do not mind the log file names as I was experimenting with different files.

```auto
docker run -d \
  --name=filebeat \
  --user=root \
  --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" \
  --volume="$(pwd)/hello.log:/usr/share/filebeat/hello.log:ro" \
  docker.elastic.co/beats/filebeat:7.10.1 filebeat -e --strict.perms=false \
  -E output.elasticsearch.hosts=["192.168.1.38:9200"]

```

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 4:18pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/15 "2022-12-29T16:18:53Z")

</div>

You are absolutely right. I needed to change the path inside filebeat.yml from this :

```auto
    - /home/raju/elk/docker/hello.log

```

To this after mounting volume.

```auto
./hello.log

```

Thank you so much. You are the best. You saved me a headache of two days and saved me a lot of time.

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 4:22pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/16 "2022-12-29T16:22:50Z")

</div>

One last request before you go. Is there a way I can set a filter so I can only see warnings and error messages from my log files instead of having logged everthing.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2022, 4:22pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/17 "2022-12-29T16:22:52Z")

</div>

No Problem... perhaps slow down a bit and think about the container's perspective...

Many posts ago I said...

> [@stephenb](#):
>
> I'm trying to give you suggestions. **I suggested that you exec into the docker container once it is running and look for the path for your log file. That is the path that has to be in the yml file that filebeat uses**

Ok now... keep going!

---

<div class="post-metadata">

**Author:** ![Raja\_Muneer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raja_muneer/32/115322_2.png) [@Raja\_Muneer](https://discuss.elastic.co/u/Raja_Muneer)\
**Post date:** [December 29, 2022, 4:24pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/18 "2022-12-29T16:24:45Z")

</div>

Thank you so much @stephenb. I cannot thank enough for your valueable time.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2022, 4:29pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/19 "2022-12-29T16:29:56Z")

</div>

> [@Raja\_Muneer](#):
>
> One last request before you go. Is there a way I can set a filter so I can only see warnings and error messages from my log files instead of having logged everthing.

That question is not clear...

The App Logs... filter with processors, ingest pipelines

The Filebeat Logs .... filter with logging levels

Which logs do you want to filter?

And the answer is yes... but perhaps you should take a look at the filebeat documentation and see if you can figure it out 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 26, 2023, 4:30pm UTC](https://discuss.elastic.co/t/how-can-i-send-custom-logs-in-a-specific-location-to-filebeat-running-inside-docker/322106/20 "2023-01-26T16:30:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.