Winlogbeat Kafka output not reading SSL certs with "wrong" line ends

Elastic Stack Beats
1

I’m setting up Winlogbeat to output to Kafka with SSL certificates used to authenticate. The certificates will be generated, rotated and supplied automatically by another system. But when trying to configure it, winlogbeat test config fails if the files have unix line ends.

output.kafka:
  enabled: true
  hosts: ["example.com:9093"]
  topic: dummy-topic
  ssl:
    certificate: C:/users/simon/downloads/simon.unix.pem
    key:         C:/users/simon/downloads/simon.unix.key

This throws an error:

Exiting: error initializing publisher: 1 error: no pem file C:/users/simon/downloads/simon.unix.pem

But if I replace the files with identical ones with Windows line ends it works fine.

get-content .\simon.unix.key | set-content simon.windows.key
get-content .\simon.unix.pem | set-content simon.windows.pem
# Update file names in config file
winlogbeat test config
Config OK

Is this expected behaviour? I would have expected it to happily read certificate files with either sort of line ends.

2

Does the user has right to access to pem and key.
Did you ran as a process or a service?

Why don't use the backslashes?

certificate: C:\users\simon\downloads\simon.unix.pem
key:         C:\users\simon\downloads\simon.unix.key
3

It generally doesn’t make much of a difference these days about which slashes you use in Windows, as most utilities seem to accept either.

But while gathering data to indicate that it wasn’t any of the things you’d suggested, I ran the 2 simon.unix.* files through a hexdump, and noticed that the pair of them have somehow got 3 non-visible non-ASCII bytes at the very start of the files, which had actually also been stripped out while doing the get-content | set-content fixing up of line ends.

Removing those non-ASCII bytes from the files, the config test then actually works fine, with either unix or windows line ends in the certificates.

So the issue’s identified, and it’s nothing to do with the Kafka output. Sorry for the noise.

2 Likes
4

You should see spec characters in Notepad++, View ->Show Symbol->Show All Characters.

Anyway, we are glad to hear that the problem has been resolved.