Elastic Docker Integration - not collecting logs

Elastic Stack Elastic Agent
1

I had been using the 'System' integration agent to consume my docker logs which are saved in the path: /var/lib/docker/containers/*/*-json.log

This has been working, but unfortunately, it was splitting up log lines which made capturing stack trace errors difficult. We looked into using the Docker integration via Elastic Fleets and it has a container log section now. We are however struggling to find documentation to help us troubleshoot or support us. I have also tried to search on the forum but due to the word 'Docker' being so popular, most of the articles refer to other things so I can't seem to find any posts about the issue I am facing.

The problem:

When I configure the Docker integration, it is able to pick up the metric stats perfectly, so I know the integration is working. When I try and pull the actual container logs, it does not return anything. I have used integrations such as Keycloak and System to succesfully pull these logs prior so I know it is possible.

With the docker container, there are 4 options to fill out.

  • Condition - I am unsure what to put here. There is a hyperlink that takes me to this page: Docker Provider | Fleet and Elastic Agent Guide [8.6] | Elastic but it does not explain what condition we should input, and why this option is optional.
  • Docker container log path - /var/lib/docker/containers/*/*-json.log - This is what we currently have and what has been working with other integrations.
  • Container parser's stream configuration - default is set to all
  • Additional parsers - default has the following but commented out. I have enabled and disabled them and they haven't done anything. Also can't find the documentation to explain what they do in regards to this integration:
- ndjson:
     target: json
     ignore_decoding_error: true
 - multiline:
     type: pattern
     pattern: '^\['
     negate: true
     match: after
  • Processors - the integration also creates a processor but there doesn't seem to be any reference to what it does.

I can't see how to get this working and I have been racking my brain, any guidance or help towards documentation, implementation steps would be greatly appreciated.

2

Any chance you're running into the same issue I am? Docker Logs keep getting dropped with tried to parse field [image] as object, but found a concrete value error - #21 by jerrac

Metrics work find for me, logs, not since 8.6.1 came out.

1 Like
3

Potentially, is there a way to rollback the agent? I'm gonna check the agent logs and see what version I'm running but I hope this is it. Currently have an open ticket with Support.

Thanks for flagging!

4

There's a work around for my issue now. Docker Logs keep getting dropped with tried to parse field [image] as object, but found a concrete value error - #36 by stephenb

As for rolling back the agent, on the free tier, I didn't see any option in Fleet to downgrade. I assume you have a subscription since you have a ticket open. Is there a downgrade option available in Fleet for you?

If there isn't, the only thing I can think of is uninstalling and reinstalling the older version. Not exactly fun if you have a lot of nodes...

Which makes me wonder if I can figure out how to make Ansible do all that kind of work for me...

Hope something I say helps at least a tiny bit. :\

5

You can't downgrade the agent version through fleet but you can remove it then just install it with the older version.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.