I had been using the 'System' integration agent to consume my docker logs which are saved in the path:
This has been working, but unfortunately, it was splitting up log lines which made capturing stack trace errors difficult. We looked into using the Docker integration via Elastic Fleets and it has a container log section now. We are however struggling to find documentation to help us troubleshoot or support us. I have also tried to search on the forum but due to the word 'Docker' being so popular, most of the articles refer to other things so I can't seem to find any posts about the issue I am facing.
When I configure the Docker integration, it is able to pick up the metric stats perfectly, so I know the integration is working. When I try and pull the actual container logs, it does not return anything. I have used integrations such as Keycloak and System to succesfully pull these logs prior so I know it is possible.
With the docker container, there are 4 options to fill out.
- Condition - I am unsure what to put here. There is a hyperlink that takes me to this page: Docker Provider | Fleet and Elastic Agent Guide [8.6] | Elastic but it does not explain what condition we should input, and why this option is optional.
- Docker container log path -
/var/lib/docker/containers/*/*-json.log- This is what we currently have and what has been working with other integrations.
- Container parser's stream configuration - default is set to all
- Additional parsers - default has the following but commented out. I have enabled and disabled them and they haven't done anything. Also can't find the documentation to explain what they do in regards to this integration:
- ndjson: target: json ignore_decoding_error: true - multiline: type: pattern pattern: '^\[' negate: true match: after
- Processors - the integration also creates a processor but there doesn't seem to be any reference to what it does.
I can't see how to get this working and I have been racking my brain, any guidance or help towards documentation, implementation steps would be greatly appreciated.