Osquery Manager does not return results from curl table

Elastic Stack Kibana
1

Hi Elastic team,

Running a query from the curl table returns an empty result. I've confirmed that other queries work and curl_certificates work as well.

SELECT * FROM curl WHERE url = 'https://www.google.com'

Is there any additional configuration that needs to be enabled for the curl table to work on OSQuery Manager?

Version Info: Elasticstack 8.2, OSQuery Manager 1.2.1

Broken curl:

image
image1304×1056 59.2 KB

Working curl_certificate:

image
image1259×1044 62.5 KB

2

If you run osquery standalone in interractive mode:

➜  elastic ./osqueryd -S
Using a virtual database. Need help, type '.help'
osquery> SELECT * FROM curl WHERE url = 'https://www.google.com';
W0518 13:35:22.938369 228421120 tls.cpp:101] Cannot read TLS server certificate(s): /var/osquery/certs/certs.pem
W0518 13:35:23.049543 228421120 curl.cpp:83] Error making request: certificate verify failed
+------------------------+--------+------------+---------------+-----------------+-------+--------+
| url                    | method | user_agent | response_code | round_trip_time | bytes | result |
+------------------------+--------+------------+---------------+-----------------+-------+--------+
| https://www.google.com | GET    | osquery    |               |                 |       |        |
+------------------------+--------+------------+---------------+-----------------+-------+--------+

You will see that it checks the the specific directory for the certs, the example was on Mac OS.
Here is the list of default paths defined in osquery for the different platforms:

That error should be in osquerybeat log.

The official osquery bundle contains, the certs.pem file, but we do not install it at the moment. We probably should and possibly override this with --tls_server_certs option to point to our private install location, need to research. There is the osquery ticket that mentions some potential "side-effect" of using this option: Cannot fetch HTTPS using curl table when using the `--tls_server_certs` flag for TLS pinning · Issue #6964 · osquery/osquery · GitHub, and it looks that the original purpose of this flag might have been different.

The short term fix at the moment is to copy the certs.pem from osquery repository osquery/certs.pem at master · osquery/osquery · GitHub or osquery official distribution package into the default path for the given platform /var/osquery/certs/certs.pem for Mac OS, for example.

3

Hi Aleksandr,

Thank you for the prompt reply and I believe your diagnosis is correct for HTTPS websites. However, I'm also seeing the save issue for HTTP websites that do not require any TLS certificate configuration. See screenshot below:

image
image2143×1750 141 KB

Best,
Han

4

For that particular url you get redirected to HTTPS url

Screen Shot 2022-05-19 at 8.20.45 AM
Screen Shot 2022-05-19 at 8.20.45 AM1978×638 86.3 KB

And then you are back to square one with TLS certs error:

osquery> select * from curl where url='http://ifconfig.me';
W0519 08:18:30.609016 371273216 tls.cpp:101] Cannot read TLS server certificate(s): /var/osquery/certs/certs.pem
W0519 08:18:30.714504 371273216 curl.cpp:83] Error making request: certificate verify failed
+--------------------+--------+------------+---------------+-----------------+-------+--------+
| url                | method | user_agent | response_code | round_trip_time | bytes | result |
+--------------------+--------+------------+---------------+-----------------+-------+--------+
| http://ifconfig.me | GET    | osquery    |               |                 |       |        |
+--------------------+--------+------------+---------------+-----------------+-------+--------+
5

Thank you for the clarification and this makes sense. Unfortunately, we won't be able to do a local copy of the certificate folder. Would bundling the official certs.pem be possible at a later Osquery Manager release?

6

Yes, we are working on bundling the certs file with our solution, hopefully for 8.3 release.

7

Perfect, thank you!

8

Thank you for letting us know!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.