Filebeat journald input failing to open a handle to the library

[Marko_Todoric]

Hello everyone,

i just managed to compile filebeat to work on armv7 with journald input (regular filestream works just fine) however when starting filebeat i get a following error message, filebeat keeps on running but with 1 input (the only one that's failing) id does nothing.

{
  "log.level": "error",
  "@timestamp": "2022-06-17T13:37:25.948+0200",
  "log.logger": "input.journald",
  "log.origin": {
    "file.name": "compat/compat.go",
    "file.line": 122
  },
  "message": "Input 'journald' failed with: input.go:130: input main.journald failed (id=main.journald)\n\tinput.go:174: failed to create reader for /var/log/journal/b81953a0ca694bcd9058a36cd89fbf9d journal (path=/var/log/journal/b81953a0ca694bcd9058a36cd89fbf9d): reader.go:114: failed to open journal directory /var/log/journal/b81953a0ca694bcd9058a36cd89fbf9d (path=/var/log/journal/b81953a0ca694bcd9058a36cd89fbf9d): unable to open a handle to the library",
  "service.name": "filebeat",
  "id": "main.journald",
  "ecs.version": "1.6.0"
}

filebeat.yml:

filebeat.inputs:
- type: journald
  enabled: true
  id: main.journald
  paths:
    - /var/log/journal/b81953a0ca694bcd9058a36cd89fbf9d
filebeat.overwrite_pipelines: false
setup.template.overwrite: false
setup.template.name: testindex-journald
setup.template.settings:
  index:
    number_of_shards: 1
    number_of_replicas: 0
setup.kibana:
output.elasticsearch:
  hosts: ["<redacted>"]
  username: <redacted>
  password: <redacted>
logging.level: warning
logging.metrics.enabled: false
logging.to_files: false
logging.to_stderr: true

I'm not sure what "unable to open a handle to the library" means. It could be because some of the missing dependency since I'm on alpine linux docker image (again - armv7 - raspberry pi 3).
So i would really appreciate any help on this.

[TiagoQueiroz]

Hi @Marko_Todoric, you're in a realm that's not officially supported, so things might get quite tricky from now on.

Under the hood the journald input uses some journald libraries (hence the need for CGO) to access the journal file. So if they're missing/are not compatible with the one used to compile the binary, you might face problems.

Aside that, Alpine Linux does not use the standard libc, so it would be good to check if your binary is fully compatible with the system you're running it.

Those two things together might be causing the unable to open a handle to the library error.

Assuming the binary works as expected, try to use journalctl (from within the container) to read the journal file, /var/log/journal/b81953a0ca694bcd9058a36cd89fbf9d in our case.

Also double check all file permissions to make sure Filebeat can read the journal file.

[Marko_Todoric]

Thanks Tiago, that pushed me enough in right direction. Alpine container regardless that is uses systemd does not appear to have journalctl. I tried installing some compat libc libraries to no avail, eventually i just switched to debian container where everything works like a charm There might be a way to make it work but i would need to spend more time investigating packages and I'm not sure that it's worth it.

Thanks a lot for your time!