Help with Docker prospector and dissect processor on filebeat

Hi,
I'm trying to use filebeat to parse my docker logs, and dissect them to send to Elasticsearch.

My filebeat.yml looks like this
filebeat.prospectors:
- type: docker
document_type: docker
containers:
ids: "*"
path: '/var/lib/docker/containers'
combine_partial: true
processors:
- add_cloud_metadata: ~
- add_docker_metadata: ~
- dissect:
field: log
tokenizer: "[%{log_stream}] [%{log_level}] [%{log_thread}] [%{@timestamp}] - "%{log_message}"\n"

A sample log line from my docker logs looks like this
{"@timestamp":"2019-03-29T16:01:19.316Z","@metadata":{"beat":"filebeat","type":"doc","version":"6.6.2"},"offset":409357,"log":{"file":{"path":"/var/lib/docker/containers/d2ada679e60d1f88be90689eaf8be3aaf062dd76b42fc8af951b9b5c52a9b7b6/d2ada679e60d1f88be90689eaf8be3aaf062dd76b42fc8af951b9b5c52a9b7b6-json.log"}},"stream":"stdout","prospector":{"type":"docker"},"input":{"type":"docker"},"meta":{"cloud":{"region":"us-east-1","availability_zone":"us-east-1c","instance_id":"i-0ddd0540e506d573e","machine_type":"t3.large","provider":"ec2"}},"message":"[logger1] [ERROR] [tomcat-http--48] [29/03/2019 12:01:19,315] - " (writeError) [some_data]Some 'error message. [somepackage/someclass(someObject.java:1354)]","source":"/var/lib/docker/containers/d2ada679e60d1f88be90689eaf8be3aaf062dd76b42fc8af951b9b5c52a9b7b6/d2ada679e60d1f88be90689eaf8be3aaf062dd76b42fc8af951b9b5c52a9b7b6-json.log","docker":{"container":{"id":"d2ada679e60d1f88be90689eaf8be3aaf062dd76b42fc8af951b9b5c52a9b7b6","labels":{"com":{"docker":{"compose":{"project":"master","service":"web","version":"1.22.0","config-hash":"f02330e2e031990958d061bd36dbc1e1b5f59ca5f356bfb76e4134010e277f96","container-number":"1","oneoff":"False"}}},"PROJECT":"web","PROJECT_NAME":"Web"},"image":"some-repo/web:master","name":"master_web_1"}},"beat":{"name":"filebeat","hostname":"filebeat","version":"6.6.2"},"host":{"name":"filebeat"}}

Nothing seems broken out, I would've thought I would've had some keys created in the JSON output to match the keys from dissect? Instead I get standard message key with everything in it.

Any help would be appricated. Thanks

Really sorry, got this working, with a simplier tokenizer string.
So the one listed here must be off.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.