Heartbeat: make http check.response.body get negative result based on the match

[lowry (Lowry_Xiao)]

Hello everyone,
Wondering if there's solution for this case:
We want to do http check over an endpoint: http://localhost:8080/health,
it returns 3 kinds of response(not json, plain text. just for example)
“status”: “green”,
“status”: “yellow”,
“status”:“red”

Idea is that:

1. HB returns down when "status":"red" in the http check response body.
2. HB returns up when "status":"green" or “status”: “yellow” in the http check response body.

However we can get it done by listing all pattern under http.check.body , like this:

check.response:
  body:
    - "status" : "green"
    - "status" : "yellow"

But if there’s another usecase needs to do the same thing, we have to list all possible patterns too.

So wondering if we can have an uniform solution for this:
From my perspective, there should be 2 ways:

1. add a new option like pattern_present under check.reponse.body or under response, the same as response.include_body to control the final match result based on the pattern matching, like this:

check.response:
   body:
        pattern_present: false
        - "status": "red"

if the value of pattern_present is true, the logic of HB is as same as orignial.
But if it is false, the check result will be success only if this pattern is not in the http check body and it will be fail when the pattern is in the body.

2. make checkBody more powerful, like makes it can support the condition like http.check.json.

Fyi, we have to think other solution since the regex in Beats doesn't support negative look:

if setup the negative regexp pattern in HB:

check.response:
status: 200
body: '((?!"status":"red").)*'

Will get this error message in HB’s log:

2020-08-06T20:26:59.506-0700 ERROR [reload] cfgfile/list.go:94 Error creating runner from config: job err error parsing regexp: invalid or unsupported Perl syntax: `(?!` accessing 'check.response.body' (source:'/ebay/beats/config/monitors.d/debug.yml')

Also can refer to this talk in stackoverflow:

Negative Look Ahead Go regular expressions

[Andrew_Cholakian1 (Andrew Cholakian)]

This is a great idea for an enhancement. I've written this up as github issue so we can better track it here: https://github.com/elastic/beats/issues/20728 . Let's move the conversation there.

[lowry (Lowry_Xiao)]

Thanks Andrew for making this happens.
Hope we can get a solution for this case.