Filebeat config to read logs using docker

Elastic Stack Beats
,
1

Am able to run the filebeat via docker , however the output says log is taken.. i dont see any data in elastic , any suggestions would be helpful.
thanks

Command User

ocker run --rm --name filebeat --volume="/usr/share/filebeat/data/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro" --volume="/var/run/docker.sock:/var/run/docker.sock:ro" --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" --network host -e -strict.perms=false container/beats/filebeat:7.5.0

Output:

2020-01-03T07:28:55.909Z INFO instance/beat.go:297 Setup Beat: filebeat; Version: 7.5.0
2020-01-03T07:28:55.909Z INFO [index-management] idxmgmt/std.go:182 Set output.elasticsearch.index to 'filebeat-7.5.0' as ILM is enabled.
2020-01-03T07:28:55.909Z INFO elasticsearch/client.go:171 Elasticsearch url: http://10.24.89.106:9200
2020-01-03T07:28:55.910Z INFO [publisher] pipeline/module.go:97 Beat name: a302-2550-4074.ldn.swissbank.com
2020-01-03T07:28:55.911Z INFO instance/beat.go:429 filebeat start running.
2020-01-03T07:28:55.911Z INFO registrar/migrate.go:104 No registry home found. Create: /usr/share/filebeat/data/registry/filebeat
2020-01-03T07:28:55.911Z INFO registrar/migrate.go:112 Initialize registry meta file
2020-01-03T07:28:55.912Z INFO registrar/registrar.go:108 No registry file found under: /usr/share/filebeat/data/registry/filebeat/data.json. Creating a new registry file.
2020-01-03T07:28:55.912Z INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2020-01-03T07:28:55.914Z INFO registrar/registrar.go:145 Loading registrar data from /usr/share/filebeat/data/registry/filebeat/data.json
2020-01-03T07:28:55.914Z INFO registrar/registrar.go:152 States Loaded from registrar: 0
2020-01-03T07:28:55.914Z INFO crawler/crawler.go:72 Loading Inputs: 1
2020-01-03T07:28:55.914Z INFO log/input.go:152 Configured paths: [/app/test/logs/app_auto*.log]
2020-01-03T07:28:55.914Z INFO input/input.go:114 Starting input of type: log; ID: 3140521193032836508
2020-01-03T07:28:55.914Z INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1

Filebeat.yaml

filebeat.inputs:

  • type: log
    enabled: true
    paths:
    • /app/test/logs/app_auto*.log

output.elasticsearch:
hosts: ["HOST:9200"]

2

@Praveen_V,

Are you sure you have the permissions needed to read the files in that directory? Can you try re-running filebeat with -d '*' and see if there's any additional debug logs?

1 Like
3

@Alex_Kristiansen . Thanks for the reply

Was able to achieve using direct Linux install of Filebeat , rather than Dockers.
However am trying to extract the time field from the log message , can you help to point me towards any link where this issue was sorted.

Sample data in the message field of Elastic : "2019-12-23 03:38 +00:00: Session not loaded"

Thanks agian

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.