XML Array parsing

[Matthias_Brauchle]

Hello,

Setup:

• Elasticsearch
• Elastic Agent with the CEL Integration (Documentation)
• XML Processor
• Ingest Pipeline

My goal is to store the output in Elasticsearch (of course).

The URL is from Cisco CUCM and responds with XML. Due to no direct Agent integration, which can handle XML (not like JSON), I use the CEL Integration to get the response into a dedicated body.

Agent config:

PUT kbn:/api/fleet/package_policies/225dd895-a7f4-4568-920f-698c9b27a26d
{
  "package": {
    "name": "cel",
    "version": "1.5.0"
  },
  "name": "GetClusterInfoList",
  "namespace": "cucm_http_ast",
  "description": "",
  "policy_id": "3ee6a5d0-5d3f-11ee-a52e-a7e9b22479c8",
  "vars": {},
  "inputs": {
    "cel-cel": {
      "enabled": true,
      "streams": {
        "cel.cel": {
          "enabled": true,
          "vars": {
            "data_stream.dataset": "cucm.ast",
            "resource_url": "https://<ipaddress>/ast/ASTisapi.dll?GetClusterInfoList",
            "resource_interval": "1m",
            "program": "get(state.url).as(body, {\n    \"events\": [\n        {\n            \"http\": {\n                \"response\": {\n                    \"body\": {\n                        \"content\": string((body).Body),\n                        \"bytes\": (body).ContentLength\n                    },\n                    \"status_code\": (body).StatusCode,\n                    \"successful\": int((body).StatusCode) == 200\n                },\n                \"request\": {\n                    \"url\": (body).Request.URL,\n                    \"method\": (body).Request.Method\n                }\n            }\n        }\n    ]\n})",
            "regexp": "#products: '(?i)(Elasticsearch|Beats|Logstash|Kibana)'\n#solutions: '(?i)(Search|Observability|Security)'\n",
            "username": "user",
            "password": "password",
            "redact_fields": [],
            "delete_redacted_fields": false,
            "resource_ssl": "verification_mode: none",
            "resource_redirect_headers_ban_list": [],
            "oauth_scopes": [],
            "processors": "- decode_xml:\n    field: http.response.body.content\n    target_field: xml\n    ignore_missing: true\n    ignore_failure: true\n    when:\n        equals:\n            http.response.successful: true\n- drop_fields:\n    fields: [\"http.response.successful\"]\n    ignore_missing: false",
            "tags": [
              "forwarded",
              "GetClusterInfoList"
            ]
          }
        }
      }
    }
  }
}

But this configuration stores output like the following as a JSON list in one document.

<GetClusterInformationListReply ErrorCode="0">
<NodeList>
<Node HostName="Node1" FQDN="Node1.domain" Role="Roles" productTypeId="1" DNSResolved="false" ASTAlive="true"/>
<Node HostName="Node2" FQDN="Node2.domain" Role="Roles" productTypeId="1" DNSResolved="false" ASTAlive="true"/>
</NodeList>
</GetClusterInformationListReply>

But I would need every line a dedicated document in Elasticsearch.

How would you achieve this? Are there any hints for the CEL input or processors? I tried it with the script processor in both - input pipeline and in the agent. But none seemed to work.

Thanks in advance!