How can i increment cursor by one for each run in httpjson

Elastic Stack Beats
1

Hi im using the httpjson input module in filebeat and im trying to achieve the following without any luck

In words:

  1. I need to fetch an API every 10s
  2. In the first run i need to set query param page = 0
  3. In the following runs i need to set query param page to page+1 if .last_response.body.meta.has_next_results equals true, else page

I tried to achieve this using a cursor and request.transform but it always goes to the default.
Below is the YAML:

  filebeat.inputs:
  - type: httpjson
    interval: 5s
    request.method: GET
    request.url: https://api.com
    cursor:
      page:
        value: '[[if eq .last_response.body.meta.has_next_results true]][[add .cursor.page 1]][[else]][[.cursor.page]][[end]]'
        default: '0'
    request.transforms:
      - set:
          target: url.params.per_page
          value: '10'
      - set:
          target: url.params.order_key
          value: 'created_at'
      - set:
          target: url.params.order_direction
          value: 'asc'
      - set:
          target: url.params.page
          value: '[[.cursor.page]]'
          default: '0'
    response.split:
      target: body.activities
      ignore_empty_value: true
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.