Hi im using the httpjson input module in filebeat and im trying to achieve the following without any luck
In words:
- I need to fetch an API every 10s
- In the first run i need to set query param page = 0
- In the following runs i need to set query param page to page+1 if .last_response.body.meta.has_next_results equals true, else page
I tried to achieve this using a cursor and request.transform but it always goes to the default.
Below is the YAML:
filebeat.inputs:
- type: httpjson
interval: 5s
request.method: GET
request.url: https://api.com
cursor:
page:
value: '[[if eq .last_response.body.meta.has_next_results true]][[add .cursor.page 1]][[else]][[.cursor.page]][[end]]'
default: '0'
request.transforms:
- set:
target: url.params.per_page
value: '10'
- set:
target: url.params.order_key
value: 'created_at'
- set:
target: url.params.order_direction
value: 'asc'
- set:
target: url.params.page
value: '[[.cursor.page]]'
default: '0'
response.split:
target: body.activities
ignore_empty_value: true