# Error "Panic in job" - Unable to monitor my service
**URL:** https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498
**Category:** Beats
**Tags:** heartbeat
**Created:** [March 28, 2020, 3:47pm UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498 "2020-03-28T15:47:35Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@luok0](https://discuss.elastic.co/u/luok0)
#### Post date: [March 28, 2020, 3:47pm UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498/1 "2020-03-28T15:47:35Z")
Got below error in heartbeat's debug log when tried to monitor one service while it works fine for other services:
```auto
hearbeat.yml:
// # Configure monitors
heartbeat.monitors:
- type: http
# List or urls to query
urls: ["https://dncpri1p.cny.net:21375/dataservices/v1/b15/eok/metadata/20200213/OD/grok.deoiO.YjiwG:name=jack,Index=pc,CvType=persen,Frequency=2H/"]
check.request:
method: GET
check.response:
status: 200
# body: '{"error" : null*}'
# Configure task schedule
schedule: '@every 10s'
ssl:
enabled: true
verification_mode: none
2020-03-28T11:09:03.005-0400 ERROR scheduler/scheduler.go:355 Panic in job 'http@https://dncpri1p.cny.net:21375/dataservices/v1/b15/eok/metadata/20200213/OD/grok.deoiO.YjiwG:name=jack,Index=pc,CvType=persen,Frequency=2H/'. Recovering, but please report this: runtime error: invalid memory address or nil pointer dereference.
2020-03-28T11:09:03.008-0400 ERROR scheduler/scheduler.go:357 Stacktrace: goroutine 167 [running]:
runtime/debug.Stack(0x10fcf57, 0x3a, 0xc0423368d8)
/usr/local/go/src/runtime/debug/stack.go:24 +0xae
github.com/elastic/beats/heartbeat/scheduler.(*Scheduler).runTask.func1.1(0xc042136af0, 0xc0424b7e80)
/go/src/github.com/elastic/beats/heartbeat/scheduler/scheduler.go:357 +0xfe
panic(0xf4ba60, 0x1817bc0)
/usr/local/go/src/runtime/panic.go:502 +0x237
github.com/elastic/beats/heartbeat/monitors/active/dialchain.TLSLayer.func1.1(0x119e540, 0xc042450a80, 0x3, 0xc0424b21a8, 0x22, 0x119e540)
/go/src/github.com/elastic/beats/heartbeat/monitors/active/dialchain/tls.go:83 +0x462
github.com/elastic/beats/heartbeat/monitors/active/dialchain.afterDial.func1(0x10bfd26, 0x3, 0xc0424b21a8, 0x22, 0x8, 0xc04238a440, 0xc0424d8d70, 0xc0424d8cf8)
/go/src/github.com/elastic/beats/heartbeat/monitors/active/dialchain/util.go:89 +0xc1
github.com/elastic/beats/libbeat/outputs/transport.DialerFunc.Dial(0xc04223eae0, 0x10bfd26, 0x3, 0xc0424b21a8, 0x22, 0x410c63, 0xc0420b4500, 0x50, 0x48)
/go/src/github.com/elastic/beats/libbeat/outputs/transport/transport.go:40 +0x55
github.com/elastic/beats/heartbeat/monitors/active/http.(*SimpleTransport).RoundTrip(0xc0421e4a50, 0xc0421fce00, 0x0, 0x0, 0x0)
/go/src/github.com/elastic/beats/heartbeat/monitors/active/http/simple_transp.go:84 +0x105
net/http.send(0xc0421fcc00, 0x1188e20, 0xc0421e4a50, 0xbf97f807b5b9dbe4, 0xb74201d71, 0x182dde0, 0xc042004138, 0xbf97f807b5b9dbe4, 0xc0424d9068, 0x1)
/usr/local/go/src/net/http/client.go:252 +0x18c
net/http.(*Client).send(0xc0421e4a80, 0xc0421fcc00, 0xbf97f807b5b9dbe4, 0xb74201d71, 0x182dde0, 0xc042004138, 0x0, 0x1, 0x3)
/usr/local/go/src/net/http/client.go:176 +0x101
net/http.(*Client).Do(0xc0421e4a80, 0xc0421fcc00, 0x182dde0, 0xc0421fcc00, 0x57fda2)
/usr/local/go/src/net/http/client.go:615 +0x294
github.com/elastic/beats/heartbeat/monitors/active/http.execRequest(0xc0421e4a80, 0xc0421fcc00, 0xc0422914c0, 0xbf97f803b5b9dbe4, 0x7ba737d71, 0x182dde0, 0x0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/elastic/beats/heartbeat/monitors/active/http/task.go:252 +0xad
github.com/elastic/beats/heartbeat/monitors/active/http.execPing(0xc0421e4a80, 0xc0421fc900, 0x0, 0x0, 0x0, 0x3b9aca000, 0xc0422914c0, 0x0, 0x0, 0x0, ...)
/go/src/github.com/elastic/beats/heartbeat/monitors/active/http/task.go:226 +0x143
github.com/elastic/beats/heartbeat/monitors/active/http.createPingFactory.func1(0xc0421e4900, 0x3, 0xc0424d96f0, 0xc0424d9758)
/go/src/github.com/elastic/beats/heartbeat/monitors/active/http/task.go:171 +0x449
github.com/elastic/beats/heartbeat/monitors.MakePingIPFactory.func1.1(0x120, 0xc0424d9778, 0x410c63)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:194 +0x38
github.com/elastic/beats/heartbeat/monitors.MakeSimpleCont.func1(0x10, 0xf81240, 0xc0424d97a8, 0x410c63, 0xc04223e8e0, 0x20)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:184 +0x2d
github.com/elastic/beats/heartbeat/monitors.funcTask.Run(0xc04213e3b0, 0x18, 0x20, 0x20, 0xc04223e8e0, 0x0, 0xc04205d000)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:440 +0x2e
github.com/elastic/beats/heartbeat/monitors.WithFields.func1(0xc0424d9701, 0xc04223e8e0, 0xc0424d97f8, 0x7cf9a8, 0xfb4920, 0xc04223e8e0)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:398 +0x51
github.com/elastic/beats/heartbeat/monitors.funcTask.Run(0xc04223e8e0, 0x118b520, 0xc04213e3b0, 0x118b520, 0xc04223e8e0, 0x182dde0, 0x1e9614)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:440 +0x2e
github.com/elastic/beats/heartbeat/monitors.makeByHostAnyIPJob.func1(0x7, 0xc0423de91b, 0x5, 0x2704b70, 0xc04209b9a8, 0x42824e)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:323 +0x940
github.com/elastic/beats/heartbeat/monitors.annotated.func1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc0420682c0, 0xc0421e58c0, 0x10c66cd, ...)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:140 +0xce
github.com/elastic/beats/heartbeat/monitors.MakeJob.func1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc0421e58c0, 0xc8, 0xc0421381a8, ...)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:127 +0xef
github.com/elastic/beats/heartbeat/monitors.(*funcJob).Run(0xc04240c560, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x582169631, 0x182dde0, ...)
/go/src/github.com/elastic/beats/heartbeat/monitors/util.go:438 +0x89
github.com/elastic/beats/heartbeat/monitors.(Job).Run-fm(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc042388420, 0xc04209bf88, 0xc04209bef0, ...)
/go/src/github.com/elastic/beats/heartbeat/monitors/task.go:120 +0x8d
github.com/elastic/beats/heartbeat/monitors.(*task).prepareSchedulerJob.func1(0xc000000010, 0x110f030, 0xc042136af0)
/go/src/github.com/elastic/beats/heartbeat/monitors/task.go:85 +0x84
github.com/elastic/beats/heartbeat/scheduler.(*Scheduler).runTask.func1(0xc042136af0, 0xc0424b7e80, 0xc042136af0, 0xc04240caa0)
/go/src/github.com/elastic/beats/heartbeat/scheduler/scheduler.go:362 +0x62
created by github.com/elastic/beats/heartbeat/scheduler.(*Scheduler).runTask
/go/src/github.com/elastic/beats/heartbeat/scheduler/scheduler.go:352 +0x63
```
---
### Author:  [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [March 30, 2020, 2:43am UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498/2 "2020-03-30T02:43:13Z")
Based on the stacktrace it looks like you're using an old version of heartbeat. Can you upgrade to 7.6.1? It has a rewritten scheduler that should not have this bug.
Out of curiosity, which version of heartbeat was this error from?
---
### Author:  [@luok0](https://discuss.elastic.co/u/luok0)
#### Post date: [March 30, 2020, 8:37am UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498/3 "2020-03-30T08:37:47Z")
Thanks for your reply.
The version is 6.5.0. We are using it because our ELK stack is based on 6.5.0 version. 7.6.1 was the version we tried with at the beginning, the dashboard can not be imported in Kibana 6.5.0.
As per your suggestion, in order to use heartbeat without being disturbed by this teething bug, seems upgrade the whole ELK to 7.6 the later version would be our only option here?
---
### Author:  [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [March 30, 2020, 1:07pm UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498/4 "2020-03-30T13:07:03Z")
Looking more closely, I see this is not a scheduler bug (the scheduler is just rescuing a panic), but rather a bug elsewhere. Would you mind upgrading heartbeat to 6.8.1 and seeing if that fixes the issue? It looks like it may.
---
### Author:  [@luok0](https://discuss.elastic.co/u/luok0)
#### Post date: [March 30, 2020, 4:13pm UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498/5 "2020-03-30T16:13:59Z")
Thanks. 6.8.1 works though, failed with response body check when I uncommented in yml.
I understand from below case that `check.response.body` is to match the entire response body. This works great for simple examples however doesn't work when we only want to find a small string in a large body. Was the enhancement introduced?
> [@Heartbeat: http check.response](https://discuss.elastic.co/t/heartbeat-http-check-response/86219):
>
> Hi, I am having an issue using heartbeat, specifically check.response.body I have the following heartbeat.yml heartbeat.monitors: - type: http # List or urls to query urls: ["http://localhost:8090/base/status.json"] # request details: check.request: method: GET check.response: body: '{"abc":"123"}' # Configure task schedule schedule: '@every 10s' It always returns: "UP": false "error": "type": "validate" "error": "message": "body mismatch" If I replace: check.response…
---
### Author:  [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [March 30, 2020, 6:25pm UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498/6 "2020-03-30T18:25:05Z")
Actually, those perform a regex match. It is case sensitive though, did you try adjusting the case?
---
### Author:  [@luok0](https://discuss.elastic.co/u/luok0)
#### Post date: [March 31, 2020, 3:30pm UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498/7 "2020-03-31T15:30:15Z")
The response would be like below, but for a valid check, only need to seek for `"error": null`.
body: '{"error" : null\*}' doesn't work as expected. How to set correctly for matching above requirements?
```auto
{
"error" : null,
"results" : [ {
"Id" : {
"id" : "46245745375425",
"metadata" : { },
"valid" : true
},
"key" : {
"Type" : "grok.DerivedMktDataUFO.YieldCurveKey",
"currently" : "U",
"cType" : "Q",
"frequency" : "2s",
"index" : "X2"
},
"objectType" : "834721935598",
"subTag" : "Ldn",
"tag" : "wfawfd0213",
"transId" : "R922ff942",
"updatedBy" : "sdsdfid",
"userId" : "adsfghgh",
"validFrom" : "2020-02-13T19:33:46.574",
"validTo" : "2030-01-01T00:00:00.000"
} ]
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 28, 2020, 3:30pm UTC](https://discuss.elastic.co/t/error-panic-in-job-unable-to-monitor-my-service/225498/8 "2020-04-28T15:30:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.