Qualys VMDR bug

djkprojects · July 9, 2024, 8:34am

Hello,

It looks that the v4.0.0 of Qualys VMDR integration is broken as when enabling Host Detection we are getting the following error:

failed to check program: failed compilation: ERROR: :15:9: unsupported syntax '?'
| ?"xml": state.keep_xml ? optional.of(string(xml)) : optional.none(),
| ........^ accessing config

Is the "?" in this line:

github.com

elastic/integrations/blob/91a6707d8e3af8854418a711f8f36fb2861a94be/packages/qualys_vmdr/data_stream/asset_host_detection/agent/stream/input.yml.hbs#L42


      
            :
              state.url
          )).with({
              "Header":{
                  "X-Requested-With": ["curl"],
                  "Authorization": ["Basic "+string(base64(state.user+":"+state.password))],
              }
          }).do_request().as(resp, resp.Body.as(xml, bytes(xml).decode_xml('qualys_api_2_0').as(body, {
              "events": body.doc.HOST_LIST_VM_DETECTION_OUTPUT.RESPONSE.HOST_LIST.HOST.map(h, 
                h.DETECTION_LIST.DETECTION.map(v, {
                  ?"xml":    state.keep_xml ? optional.of(string(xml)) : optional.none(),
                  "message": h.with({"DETECTION_LIST": v}).encode_json(),
                })
              ).flatten(),
              "url": (
                has(body.doc.HOST_LIST_VM_DETECTION_OUTPUT.RESPONSE.WARNING) && has(body.doc.HOST_LIST_VM_DETECTION_OUTPUT.RESPONSE.WARNING.URL)
                ?
                  body.doc.HOST_LIST_VM_DETECTION_OUTPUT.RESPONSE.WARNING.URL
                :
                  state.url.parse_url().Scheme + "://" + state.url.parse_url().Host
              ),

necessary?

kcreddy · July 9, 2024, 9:16am

Hey @djkprojects

Is the "?" in this line necessary:

Yes, it defines an optional type in CEL, an underlying filebeat input used to ingest the data.

It looks that the v4.0.0 of Qualys VMDR integration is broken as when enabling Host Detection we are getting the following error:

Did this error occur when you were upgrading from a previous version of the integration? If so, what was the previous version of the integration?

The optional type is also introduced in v8.12.0 Elastic Stack. Can you also confirm the version of Elastic Agent you currently setup for the integration?

Elastic Agent version can be found by navigating to Management -> Fleet -> Agents and check Version.

djkprojects · July 17, 2024, 1:28pm

Hello @kcreddy

We upgraded the agent to 8.13.4 and the integration works only partially.

for Host detection we get logs but sometimes it fails with error.message:

[
  failed eval: ERROR: <input>:12:19: no such key: HOST_LIST_VM_DETECTION_OUTPUT
   | }).do_request().as(resp, resp.Body.as(xml, bytes(xml).decode_xml('qualys_api_2_0').as(body, {
   | ..................^,
  Processor json with tag json_message in pipeline logs-qualys_vmdr.asset_host_detection-4.0.0 failed with message: field [message] not present as part of path [message]
]

For user activity it fails for each request with:

[
  failed eval: ERROR: <input>:14:21: csv: record on line 2: wrong number of fields
   |   }).do_request().as(resp,
   | ....................^,
  Processor 'conditional' with tag 'fail_on_cel_error' failed with message 'CEL program returned an error. Skipping ingest pipeline execution.'
]

Thank you

kcreddy · July 18, 2024, 9:39am

There were few bug fixes and improvements in recent Qualys integration version 4.1.1. Can you confirm if you are able to fix above issues by upgrading to latest version?

If not, it would be nice to have the sanitized API response by removing sensitive data. You can do so with Enable request tracing integration option. This captures the API request/responses to Elastic Agent's logs folder.