# \[heartbeat\]add feature to enable publisher processor or other processors' debug log for just specific targets

**URL:** <https://discuss.elastic.co/t/heartbeat-add-feature-to-enable-publisher-processor-or-other-processors-debug-log-for-just-specific-targets/281374>\
**Category:** Beats\
**Tags:** heartbeat\
**Created:** [August 13, 2021, 2:45pm UTC](https://discuss.elastic.co/t/heartbeat-add-feature-to-enable-publisher-processor-or-other-processors-debug-log-for-just-specific-targets/281374 "2021-08-13T14:45:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![lowry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lowry/32/77489_2.png) [@lowry](https://discuss.elastic.co/u/lowry)\
**Post date:** [August 13, 2021, 2:45pm UTC](https://discuss.elastic.co/t/heartbeat-add-feature-to-enable-publisher-processor-or-other-processors-debug-log-for-just-specific-targets/281374/1 "2021-08-13T14:45:13Z")

</div>

Hey Elastic friends,

Obviously, there’s some basic and amazing debug features in Elastic heartbeat, however we met a user case these days that current debug features cannot handle it.  
Given this example, since we have over 30k targets need to be monitored, in some case, we just want to observe the debug log of publisher processor for just some specific targets. That means we want to make sure the check for this target is really executed and the metrics have been sent out.

Compared to the current features to do that:

1. starting heartbeat with `-d "*"`, this prints all debug logs, that’s noisy and hard to find the log of our desired target, also uses too many disk. By the way, this still need a restart of heartbeat, this could be tough if it’s in use for production.

2. starting heartbeat with `-d "publisher"` , this also prints too much useless logs and needs a restart of heartbeat.

Wondering if there’s an option that can be setup in target file scope, like the following:

```auto
- type: http
  schedule: 36 * * * * * *
  hosts:
   - http://127.0.0.1/readiness_api
  check.request:
    method: GET
  check.response:
    status:
    - 200
    - 201
  fields:
    metadata:
      debugOn: true

```

Heartbeat will record the publisher or all processors’ debug log for this target with `debugOn: true ` option.  
This needn’t restart heartbeat and can be specified to the desired targets as you wish.

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [August 30, 2021, 6:33pm UTC](https://discuss.elastic.co/t/heartbeat-add-feature-to-enable-publisher-processor-or-other-processors-debug-log-for-just-specific-targets/281374/2 "2021-08-30T18:33:16Z")

</div>

Apologies for the delay @lowry , we somehow missed this.

Which debug data are you looking for? The `debugf` calls in the go code really can only go to the CLI, but if you let us know exactly what data you're looking for we can look into adding it as a first class feature.

---

<div class="post-metadata">

**Author:** ![lowry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lowry/32/77489_2.png) [@lowry](https://discuss.elastic.co/u/lowry)\
**Post date:** [August 31, 2021, 1:20am UTC](https://discuss.elastic.co/t/heartbeat-add-feature-to-enable-publisher-processor-or-other-processors-debug-log-for-just-specific-targets/281374/3 "2021-08-31T01:20:19Z")

</div>

@Andrew_Cholakian1 No worries.  
Actually, I've made an implement in my local branch.  
Wondering if I can file a PR then we could discuss there?  
Thanks.

---

<div class="post-metadata">

**Author:** ![lowry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lowry/32/77489_2.png) [@lowry](https://discuss.elastic.co/u/lowry)\
**Post date:** [September 16, 2021, 8:26am UTC](https://discuss.elastic.co/t/heartbeat-add-feature-to-enable-publisher-processor-or-other-processors-debug-log-for-just-specific-targets/281374/4 "2021-09-16T08:26:56Z")

</div>

@Andrew_Cholakian1  
Hi Andrew, filed a PR in beats repo:

> <https://github.com/elastic/beats/pull/27964>
>
> \## Enhancement
> 
> \## What does this PR do?
> Add a new feature to print logs in a… specific step(currently, we only focus on \`publisher\` step) of dealing with a target having the new option.
> 
> \## Why is it important?
> 
> Given this example, if there're more than 30k targets need to be monitored on a single heartbeat instance, in some case, we just want to know the debug log related to publisher processor for some specific targets. That means we're going to make sure if the check for some targets are really executed and the metrics have been sent out.
> 
> The current ways we have:
> 1. starting heartbeat with -d "\*", this prints all debug logs, that’s noisy and hard to find the log we cared, also uses too many disk. By the way, this also needs a heartbeat restart, this could be tough if it’s in use for production.
> 
> 2. starting heartbeat with -d "publisher" , this also prints too much useless logs from undesired targets and also needs a heartbeat restart.
> 
> Wondering if we could support a new option like \`debugOn\` here for a specific target, if it's \`true\`, heartbeat prints all \`publisher\` related logs out.
> 
> \`\`\`
> \- type: http
> schedule: 36 \* \* \* \* \* \*
> hosts:
> - http://127.0.0.1/readiness\_api
> check.request:
> method: GET
> check.response:
> status:
> - 200
> - 201
> fields:
> metadata:
> debugOn: true
> \`\`\`
> 
> This needn’t restart heartbeat and can be specified to the desired targets as you wish.
> 
> \## Checklist
> 
> \- \[x\] My code follows the style guidelines of this project
> \- \[x\] I have commented my code, particularly in hard-to-understand areas
> \- \[\] I have made corresponding changes to the documentation
> \- \[\] I have made corresponding change to the default configuration files
> \- \[\] I have added tests that prove my fix is effective or that my feature works
> \- \[\] I have added an entry in \`CHANGELOG.next.asciidoc\` or \`CHANGELOG-developer.next.asciidoc\`.
> 
> \## Author's Checklist
> 
> \- \[\]
> 
> \## How to test this PR locally
> 
> 
> \## Related issues
> A coversation for this feature is here:
> https://discuss.elastic.co/t/heartbeat-add-feature-to-enable-publisher-processor-or-other-processors-debug-log-for-just-specific-targets/281374
> 
> \## Use cases
> 
> Please refer to the \`Why is it important?\` part.
> 
> \## Screenshots:
> Will get the logs like the following for the example in \`Why is it important\`:
> \`\`\`
> 2021-08-20T09:01:22.013Z	INFO	\[processors\]	processing/processors.go:204	Publish event: {
> "@timestamp": "2021-08-20T09:01:22.000Z",
> "@metadata": {
> "beat": "heartbeat",
> "type": "\_doc",
> "version": "8.0.0",
> "ingress": \[
> {
> "namespace": "uptime",
> "name": "up",
> "dimensions": \[
> {
> ...skip...
> {
> "name": "scheme",
> "value": "http"
> },
> {
> "name": "url",
> "value": "http://127.0.0.1/readiness\_api"
> }
> \],
> "timestamp": "2021-08-20T09:01:22.000Z",
> "data": 1
> },
> ...skip...
> \`\`\`

Can you pls comment in there when you get a chance?  
Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 14, 2021, 10:27am UTC](https://discuss.elastic.co/t/heartbeat-add-feature-to-enable-publisher-processor-or-other-processors-debug-log-for-just-specific-targets/281374/5 "2021-10-14T10:27:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.