Couldn't Harvest log file from Docker

Elastic Stack Beats
,
1

Hi All,
I am beginner for ELK stack, Below my work scenario

  • I am going to centralize logs from my Microservices

  • Also i used ELK stack and i already config it in docker-compose.yml

  • Kibana,Elasticsearch and Logstash are completly work without any Errors

  • When i was run FileBeat locally without docker it works( i can see logs in Kibana through the logstash and Elasticsearch )

  • But when i was going to read logs file from docker mounted file using file beat it doesn't work ("filebeat":{"harvester":{"open_files":0,"running":0}})

  • i have created docker volume with my local logs folder and filebeat container folder("$PWD/log:/usr/share/filebeat/newlogs")

This is my Docker-compose File
</>
filebeat:

image: docker-logs-elk/filebeat:8.1.0

build:

  context: filebeat/

  args:

    ELK_VERSION: 8.1.0

volumes:

  - "$PWD/log:/usr/share/filebeat/newlogs"

  - "/var/run/docker.sock:/var/run/docker.sock"

depends_on:

  - elasticsearch

  - logstash

</>

below is my filebeat.yml file
</>
filebeat.inputs:

  • type: docker

    combine_partial: true

    containers:

    path: "/usr/share/filebeat/newlogs"

    stream: "stdout"

    ids:

    - "*"

    exclude_files: ['.gz$']

    ignore_older: 10m

filebeat.config:

modules:

path: '${path.config}/modules.d/*.yml'

reload.enabled: true

filebeat.autodiscover:

providers:

- type: docker

  hints.enabled: true

output.logstash:

hosts: ["logstash:5044"]
</>

Below Filebeat's Logs

</>
{"log.level":"info","@timestamp":"2022-03-25T07:24:08.510Z","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":184},"message":"Non-zero metrics in the last 30s","service.name":"filebeat","monitoring":{"metrics":{"beat":{"cgroup":{"cpuacct":{"total":{"ns":13430800}},"memory":{"mem":{"usage":{"bytes":4096}}}},"cpu":{"system":{"ticks":1130,"time":{"ms":16}},"total":{"ticks":2540,"time":{"ms":21},"value":2540},"user":{"ticks":1410,"time":{"ms":5}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral_id":"310c58ed-481b-48e1-8b12-fef53a23f6b8","uptime":{"ms":2520280},"version":"8.1.0"},"memstats":{"gc_next":17308240,"memory_alloc":9508944,"memory_total":79589344,"rss":121122816},"runtime":{"goroutines":22}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0}},"pipeline":{"clients":0,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.25,"15":0.42,"5":0.28,"norm":{"1":0.0313,"15":0.0525,"5":0.035}}}},"ecs.version":"1.6.0"}}
</>
If you have any solution.
Sincerely appreciate your time and consideration.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.