Full body response to elasticsearch

I am trying to figure out how to index the full body response into elastics using Heartbeat.

I found this: Send body response to Elastic
And it is now solved, but seems to only be during errors.

I am trying to put up a monitor of different API servers and this data is a key factor to make sure the API provides the intended requests.

Do I need to edit the heartbeat template, or some processor?
I'm pretty new to elasticsearch and I'd love any help here.

Thanks
Anders

Which version are you running? The code has been merged, but it won't be released until 7.4.0.

Hey, currently running 7.0.0

I'm sorry but you'll have to wait till 7.4.0 is released and then upgrade to that.

Does anyone have this working? I'm using heartbeat 7.4.0 with the following config, however the http.response.body.content field in Elastic is empty, and interestingly the http.response.body.bytes is 0:

heartbeat.monitors:
- type: http
  urls: ["http://localhost:5000/status"]
  schedule: '@every 30s'
  response.include_body: always
  response.include_body_max_bytes: 2048
  check.request.method: GET
  check.response:
    status: 200
    json:
      - description: check status
        condition:
          equals:
            status: running
  processors:
    - decode_json_fields:
        fields: ["http.response.body.content"]

A curl to the /status endpoint returns:

{"name":"MODEL_SERVER","status":"running","version":"1.3.3"}

What I would like to do is add the "version" JSON field and value into the document in Elastic.

The check response above works with the JSON condition "status" = "running", so the JSON object is there and being parsed. All I would like is to add that JSON object into a root field. :slight_smile:

thanks!

Hi,

I am running into the exact same issue. The decode_json_fields is not triggered. i can see the body in the discover tab and I see the entire json document there as a single string.

This is a confirmed bug and should be fixed in the upcoming 7.5 release.

1 Like

To add more detail here. When this feature was merged it contained a bug where validating the body would read the buffer and leave 0 bytes remaining for a second stage here we would excerpt the body and record its size.

This was fixed in https://github.com/elastic/beats/pull/14310 which is targeted for the 7.5 release.