When I tried the bash script you provided, which I see just follows the directions on Kafka docs, I got the same error in filebeat.
Exiting: error initializing publisher: crypto/tls: failed to parse private key
Looking into it I noticed that the script generated DSA keys for the server and client rather than RSA keys. The generated key worked okay from openssl but failed from filebeat. From my tests it looks like filebeat does not support DSA keys and will fail if one is used in either the client or the server.
Additionally, as Steffen pointed out, when generating the server certificate I needed to provide the exact Kafka hostname that was going to be used by the client. Possibly adding
insecure
to the tls options in the Kafka output config might make filebeat accept a mismatch on the hostname and cert CN. Refer to tls config docs for information on the insecure option but one should not use insecure outside of testing due to potential man-in-the-middle attacks.
I created a new bash script to create a self-signed RootCA, create the required server and client certificates, keystores, truststores and these worked for me. Will provide the script on a separate reply as it will be too large to add here.