Elastic Agent Integration - Duo

Elastic Stack Elastic Agent
1

I'm trying to use the Duo integration in Elastic Agent. In standalone mode.
The issue is that it is not using the last event published time so it is using the default value and re-pulling all Duo events. After much troubleshooting, I have determined that the variables used are not working.
The basic code is this:

inputs:
  - id: httpjson-cisco_duo
    name: cisco_duo
    revision: 1
    type: httpjson
    use_output: default
    meta:
      package:
        name: cisco_duo
        version: 1.23.0
    data_stream:
      namespace: default
    streams:
      - id: httpjson-cisco_duo.auth
        data_stream:
          dataset: cisco_duo.auth
          type: logs
        config_version: 2
        interval: 1m
        request.method: GET
        request.url: https://api-redacted.duosecurity.com/admin/v2/logs/authentication
        request.transforms:
          - set:
              target: url.params.limit
              value: 1000
          - set:
              target: url.params.sort
              value: 'ts:asc'
          - set:
              target: url.params.mintime
              value: '[[.cursor.last_published]]'
              default: >-
                [[div (((now (parseDuration "-2m")).Add (parseDuration
                "-1m")).UnixNano) 1e6]]
          - set:
              target: url.params.maxtime
              value: '[[div ((now (parseDuration "-2m")).UnixNano) 1e6]]'
          - set:
              target: header.Date
              value: '[[formatDate (now) "Mon, 02 Jan 2006 15:04:05 -0700"]]'
          - set:
              target: header.Authorization
              value: >-
                [[sprintf "Basic %s" (base64Encode (sprintf "%s:%s"
                "redacted" (hmac "sha1"
                "redacted" (formatDate (now)
                "Mon, 02 Jan 2006 15:04:05 -0700") "\n" "GET" "\n" .url.Host
                "\n" "/admin/v2/logs/authentication" "\n" .url.RawQuery)))]]
        response.split:
          target: body.response.authlogs
          ignore_empty_value: true
        cursor:
          last_published:
            value: '[[mul (toInt .last_event.timestamp) 1000]]'
        tags:
          - cisco_duo-auth
        publisher_pipeline.disable_host: true

The problem is that .cursor.last_published is not working. I can pipe it to a field with
fields:
TESTLASTPUBLISHED: '[[.cursor.last_published]]'
and its value is just literally '[[.cursor.last_published]]'.

I don't understand the details of Agent enough to troubleshoot this further. Any ideas? Could I keep track of the last published date in a file instead, and reference that file in this config? If so, how?

2

Can you try to update to the last version and see if this still happens?

The last version of this integration is 2.2.5, there were a lot of changes on it.

You could also enable the request tracer to see what timestamp is being used as the url.params.mimtime.

But I'm not sure how you do it in standalone.

3

How does one update an integration?

4

I think you need to upgrade the integration assets in Kibana and then get the new config in the integrations page.

I do not use standalone agents, only fleet managed, so I'm not sure.

5

Ah, you've gotten me on a path I didn't even think about. I am using Kibana 8.12 which maxes out the Duo integration at 1.23.0. I am going to try to upgrade my Kibana to the newest version that supports new Duo integrations and see if the configs are different.
Thanks

6

You will need to upgrade the entire stack.

7

Yeah, no problem, its just a dev server just to run Fleet to get the configs I need for standalone.

Testing now. The config is very different so here's hoping...

8

Tada. Newest version works great.
Thanks for the help. I didn't know to look for newer versions of Agent integrations. I do now.