Problem using chain within httpjson input

Elastic Stack Beats
1

Dear community,

as soon as we configure a chain task within the httpjson input we receive the following error

{"function":"github.com/elastic/beats/v7/x-pack/filebeat/input/httpjson.run.func1","file.name":"httpjson/input.go","file.line":178},"message":"Error while processing http request: error while getting keys: parsing error: \t - 1:1 unexpected EOF while scanning extensions"

When leaving out the chain step, and just use 1 request, everything is just fine.

Although we are using 8.14.2 we find it awkward the logs are mentioning v7 within the logs?

filebeat.yml

filebeat.inputs:
- type: httpjson
  enabled: true
  #request.tracer.filename: /var/log/filebeat_api_debug
  interval: 7s
  request.url: https://w.x.y.z/token/
  request.retry.max_attempts: 3
  request.timeout: 10s
  request.method: POST
  request.ssl:
    - enabled: false
  request.transforms:
    - set:
        target: header.Authorization
        value: 'Basic xxxx'
  chain:
  - step:
        request.url: https://w.x.y.z/endpoint1/
        request.method: GET

Filebeat version: 8.14.2

2

I think I solved the problem by using the required key 'replace'

3

could you kindly share your fix as am having same issue/error when using a chain.

- id: httpjson-wls-jvm-all
  name: weblogic-jvm-all
  type: httpjson
  use_output: default
  data_stream:
    namespace: default

  streams:
  - id: httpjson-wls-jvm-all
    data_stream:
      dataset: weblogic.jvm
    interval: 60s
    logging:
      level: debug

    auth.basic.user: "weblogic"
    auth.basic.password: "weblogic1980"

    # 1) Discover servers (GET items with name/state/links)
    request:
      url: "http://xxx.xxx.x.xx:7001/management/weblogic/latest/domainRuntime/serverRuntimes"
      method: GET
      transforms:
      - set: { target: header.X-Requested-By, value: elastic-agent }
      - set: { target: header.Accept, value: application/json }
      params:
        # IMPORTANT: do not use links=all here
        fields: "name,links,state"

    response:
      decode:
        target: body
        type: json
        overwrite_keys: true
      split:
        target: body.items # each event now has body.name, body.state, body.links[...]
      keep_original: true

    # 2) For each server, call its JVMRuntime
    chain:
    - step:
        request:
          # links[0].href exists after the split above
          url: "{{body.links.0.href}}/JVMRuntime"
          method: GET
          transforms:
          - set: { target: header.X-Requested-By, value: elastic-agent }
          - set: { target: header.Accept, value: application/json }
          auth.basic.user: "weblogic"
          auth.basic.password: "weblogic1980"
          params:
            links: "none"
            fields: "uptime,heapSizeCurrent,heapFreeCurrent,heapFreePercent,processCpuLoad"

        response:
          decode:
            target: weblogic.jvm # parsed JVM metrics land here
            type: json
            overwrite_keys: true
          keep_original: true
    # 3) After chain: copy the server name into a stable field
    processors:
    - script:
        lang: javascript
        source: >
          function process(event){
            var name = event.Get("body.name");
            if (name) event.Put("weblogic.server.name", name);
          }- id: httpjson-wls-jvm-all
  name: weblogic-jvm-all
  type: httpjson
  use_output: default
  data_stream:
    namespace: default

  streams:
  - id: httpjson-wls-jvm-all
    data_stream:
      dataset: weblogic.jvm
    interval: 60s
    logging:
      level: debug

    auth.basic.user: "weblogic"
    auth.basic.password: "weblogic1980"

    # 1) Discover servers (GET items with name/state/links)
    request:
      url: "http://192.168.0.54:7001/management/weblogic/latest/domainRuntime/serverRuntimes"
      method: GET
      transforms:
      - set: { target: header.X-Requested-By, value: elastic-agent }
      - set: { target: header.Accept, value: application/json }
      params:
        # IMPORTANT: do not use links=all here
        fields: "name,links,state"

    response:
      decode:
        target: body
        type: json
        overwrite_keys: true
      split:
        target: body.items # each event now has body.name, body.state, body.links[...]
      keep_original: true

    # 2) For each server, call its JVMRuntime
    chain:
    - step:
        request:
          # links[0].href exists after the split above
          url: "{{body.links.0.href}}/JVMRuntime"
          method: GET
          transforms:
          - set: { target: header.X-Requested-By, value: elastic-agent }
          - set: { target: header.Accept, value: application/json }
          auth.basic.user: "weblogic"
          auth.basic.password: "weblogic1980"
          params:
            links: "none"
            fields: "uptime,heapSizeCurrent,heapFreeCurrent,heapFreePercent,processCpuLoad"

        response:
          decode:
            target: weblogic.jvm # parsed JVM metrics land here
            type: json
            overwrite_keys: true
          keep_original: true
    # 3) After chain: copy the server name into a stable field
    processors:
    - script:
        lang: javascript
        source: >
          function process(event){
            var name = event.Get("body.name");
            if (name) event.Put("weblogic.server.name", name);
          }