Possible bug with Elastic Agent ca certificate checks

Elastic Security
1

Hi there,
TL;DR: The endpoint agent can't talk to ES as it doesn't trust the self signed ca cert, despite this being in the trusted root store of the endpoint.

The workaround: The agent requires a path to be defined in the 'outputs' section (right at the end) of the action_store.yml file that needs to be added as: ssl.certificate_authorities. However this user added path gets wiped out with each new configuration deployment.

E.g. (I copied the ca cert to the agent folder, but it can live anywhere):
ssl.certificate_authorities: ["C:\\Program Files\\Elastic\\Agent\\elasticsearch-ca.pem"]

The possible bug: There's an 'advanced' config option in the Endpoint Security integration for windows called windows.advanced.elasticsearch.tls.ca_cert but no matter what I put there (it wants a path to the ca.pem file), this seems to have no impact.

Also - it would be helpful to have this option available in the Elastic Agent integration rather than the Endpoint Security one, as this integration may not be included in a policy. The Agent integration will always be present.

If I've missed a step that tells the Elastic Agent where to find the root ca cert, then do please let me know, as that will also solve this issue.

Thank you!

2

Hello John, thanks for using our Endpoint!

It sounds like you might be editing the yaml file on disk since it gets wiped out with each new configuration update.

There is a way to set that configuration option so it's included in each configuration update.

In Kibana, go to the Fleet page from the main menu fly-out. From the Fleet Overview page, click on the Settings link in the upper right.

On that fly-out, there should be a section called Elasticsearch output configuration. You can add yaml settings in here that get applied to the Elasticsearch output section of all deployed agents and endpoints.

Note: that does not apply to the Kibana connection, which is specified in a different section that is not currently editable in the Fleet settings. Endpoint uses a connection to Kibana to download artifacts like alert exception lists.

We are working on some efforts to hopefully make it a bit easier to configure settings when people are using self-signed certificates.

2 Likes
3

Hi Doug,
Thank you for the quick repsonse!

That's perfect and worked a treat - that's a very handy option to know.
The only thing to keep in mind is that escaped backslashes will be stripped, so you need to double escape so everything works.

I added an entry like this in the Fleet settings area you suggested:
ssl.certificate_authorities: ["C:\\\\Program Files\\\\Elastic\\\\Agent\\\\elasticsearch-ca.pem"]

and it works perfectly.

Thank you!

1 Like
4

Hello!

I have same problems with Linux.
I try write to Fleet settings:
ssl.certificate_authorities: ["C:\\\\Program Files\\\\Elastic\\\\Agent\\\\elasticsearch-ca.pem", "/etc/pki/ca-trust/source/anchors/ca.crt"]
but it is not working.
What needs to be written to the Fleet settings for both windows and linux agents to run at the same time?

5

Hi there,
I'm not sure about the linux settings - I'm going to look at this over the weekend.
I did find out that I could change the Windows setting to just:
ssl.certificate_authorities: ['C:\Program Files\Elastic\Agent\elasticsearch-ca.pem']
(so no need to escape the '\' charachter) and it works perfectly.

If you look at the actions.yml file in a Linux deployment, is there an ssl section present?
If so, perhaps you can use that syntax to tell Fleet which is Windows and which is Linux?

6

Hi!

Where is actions.yml located? Elastic-agent directory consists of elastic-agent.yml and fleet.yml.
And elastic-agent log has error State changed to FAILED: 1 error: 1 error: open C:\Program Files\Elastic\Agent\ca.crt: no such file or directory reading.
I assume it is need to separate Fleet config for Windows and Linux. But I don't know how

7

Hello kmz161, thanks for using the Agent!

We may not be handling file paths for different OSes properly in the certificate_authorities list. We will take a look and put in an issue if need be.

However, you can specify the server's certificate itself in the Fleet Settings, which should work for both Linux and Windows.

An example (with a mostly omitted cert):

ssl:
  certificate_authorities:
  - |
    -----BEGIN CERTIFICATE-----
    MIIDSjCCAjKgAwIBAgIVAMwXQ/LfkLJancDaRWpAtSXc1MHWMA0GCSqGSIb3DQEB
    <text omitted>
    gST8PIDRw50XyXuRzhFg3dpcGwxirOeW3+QGn2bgDGxMiddkLJqguGMZuuHTNflq
    c3mPv8tjuS+firXMklF8vL8UCpd1GQLoxawBqBbw
    -----END CERTIFICATE-----

There's also a verification_mode setting under ssl that can be set to full for full certificate verification, or certificate to verify the certificate but ignore the hostname matching the certificate's CN value.

Documentation link: Configure SSL | Filebeat Reference [7.12] | Elastic

1 Like
8

Hello!
Thanks for answer!

I try to use advice, but I have error
sub_type: 'FAILED' message: Application: filebeat--7.12.0[2a74db10-8d55-11eb-8dda-c7582ac61c86]: State changed to FAILED: 1 error: 1 error: file is not a certificate adding inline to the list of known CAs
I add certificate to /etc/pki/ca-trust/source/anchors, but problem doesn't solved

9

Hello kmz161.

I've seen a similar error when I didn't have the certificate entered properly in that text box. Make sure to have the | character on the first line of the yaml entry. Also make sure the certificate lines are all indented the proper number of spaces.

Another example of this entry is here: Configure SSL | Filebeat Reference [7.12] | Elastic

As for adding the certificate to /etc/pki/ca-trust/source/anchors, did you then run update-ca-trust? I believe that is needed on RedHat/Centos systems to fully update the certificate trust settings.

1 Like
10

Hello, Doug Weyrauch!

My issue resolved!
Thank you!

1 Like
11

Hi Doug,
This solution has worked really well on both Mac and Linux endpoints - problem solved.

Many thanks.

1 Like
12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.