# Heartbeat: Ping TImeout on hosts causes state.duration\_ms to stay at 0? **URL:** \ **Category:** Beats\ **Tags:** heartbeat\ **Created:** [May 22, 2023, 9:09pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051 "2023-05-22T21:09:32Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![scantron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scantron/32/114018_2.png) [@scantron](https://discuss.elastic.co/u/scantron)\ **Post date:** [May 22, 2023, 9:09pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/1 "2023-05-22T21:09:33Z")
We are monitoring a few hosts using ICMP on heartbeat. Heartbeat accurately depicts the uptime using the state.duration\_ms field. However, shutting down a host for testing leads to the error.message field of "ping timeout", which is fine, but whenever this error message occurs, the state.duration\_ms field stays at 0. The state.id also updates on every event, meaning all of the state subfields are constantly reset, and do not track from when the host first went down. We want to be able to track for how long the host has been down for using that state.duration\_ms field, but we can't because Heartbeat considers each ping timeout to be a new state, and thus the duration\_ms field does not properly track. Is there anything we can do to achieve this? ---
**Author:** ![jkambic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jkambic/32/53306_2.png) [@jkambic](https://discuss.elastic.co/u/jkambic)\ **Post date:** [May 23, 2023, 2:50pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/2 "2023-05-23T14:50:04Z")
Hi @scantron, sorry to hear you're having issues using Heartbeat. This is a feature that has undergone numerous iterations throughout v8.x. Could you please let me know what version of the Elastic Stack you are running? ---
**Author:** ![scantron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scantron/32/114018_2.png) [@scantron](https://discuss.elastic.co/u/scantron)\ **Post date:** [May 23, 2023, 4:52pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/3 "2023-05-23T16:52:39Z")
I am running the latest, 8.7 ---
**Author:** ![jkambic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jkambic/32/53306_2.png) [@jkambic](https://discuss.elastic.co/u/jkambic)\ **Post date:** [May 24, 2023, 2:31pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/4 "2023-05-24T14:31:23Z")
Thanks for sharing that - so far I haven't been able to reproduce this issue on my side. Some additional things that could help diagnose the issue: - Are you able to include any config for your monitor? - It is odd that you're seeing a different state ID on each ping. Are you using run once mode? ---
**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\ **Post date:** [May 24, 2023, 2:47pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/5 "2023-05-24T14:47:14Z")
It may also be helpful to take a look at the logs, are any errors showing up? ---
**Author:** ![scantron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scantron/32/114018_2.png) [@scantron](https://discuss.elastic.co/u/scantron)\ **Post date:** [May 24, 2023, 4:56pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/6 "2023-05-24T16:56:14Z")
Hi J, thanks for your response. Here is a sample config for the host that is having this problem. Under the monitors.d folder, we have this config enabled: ```plaintext id: sampleid123 name: samplename123 enabled: true schedule: '@every 10s' hosts: [ "the-problem-host" ] ipv6: true mode: any timeout: 5s wait: 1s tags: ["tag1"] fields_under_root: true fields: field1: value1 ``` Names and fields have been swapped out, but all options are the same. Here are two sample logs that demonstrate this behavior in JSON format: Log 1 ```auto { "_index": "heartbeat_index", "_id": "hCwHTogB0CLBbzCvXggs", "_version": 1, "_score": null, "_source": { "error": { "type": "io", "message": "ping timeout" }, "hostname": "the-problem-host" }, "state": { "down": 1, "duration_ms": "0", "up": 0, "ends": { "status": "up", "duration_ms": "0", "up": 1, "ends": null, "down": 0, "checks": 1, "flap_history": null, "id": "default-1884e074654-6e51", "started_at": "2023-05-24T13:51:19.636833008Z" }, "status": "down", "id": "default-1884e0759dc-6e52", "flap_history": [], "started_at": "2023-05-24T13:51:24.636739175Z", "checks": 1 }, "summary": { "up": 0, "down": 1 }, "monitor": { "duration": { "us": 5002514 }, "name": "samplename1", "timespan": { "gte": "2023-05-24T13:51:24.636Z", "lt": "2023-05-24T13:51:34.636Z" }, "check_group": "0cb1a619-fa3a-11ed-a6b6-00163e77ee12", "status": "down", "ip": "redacted", "type": "icmp", "id": "sampleid1" }, "field1":"value1", "@version": "1", "agent": { "type": "heartbeat", "ephemeral_id": "4773b85c-53b9-43af-b353-1777ec80b0d9", "id": "c51f1f5c-afbf-458b-a0aa-feb5ae48312b", "name": "heartbeat_instance", "version": "8.7.0" } }, "fields": { "@timestamp": [ "2023-05-24T13:51:19.634Z" ], "monitor.timespan.lt": [ "2023-05-24T13:51:34.636Z" ], "state.ends.started_at": [ "2023-05-24T13:51:19.636Z" ], "state.started_at": [ "2023-05-24T13:51:24.636Z" ], "monitor.timespan.gte": [ "2023-05-24T13:51:24.636Z" ] } ``` Log 2 ```auto { "_index": "heartbeat_index", "_id": "rSwHTogB0CLBbzCvbQh_", "_version": 1, "_score": null, "_source": { "error": { "type": "io", "message": "ping timeout" }, "hostname": "the-problem-host" }, "state": { "checks": 1, "duration_ms": "0", "up": 0, "ends": { "status": "up", "duration_ms": "0", "up": 1, "ends": null, "down": 0, "checks": 1, "flap_history": null, "id": "default-1884e0755a7-6cbf", "started_at": "2023-05-24T13:51:23.559937829Z" }, "down": 1, "id": "default-1884e07692f-6cc0", "flap_history": [], "started_at": "2023-05-24T13:51:28.559665451Z", "status": "down" }, "summary": { "up": 0, "down": 1 }, "monitor": { "duration": { "us": 5001712 }, "name": "samplename1", "timespan": { "gte": "2023-05-24T13:51:28.559Z", "lt": "2023-05-24T13:51:38.559Z" }, "check_group": "0f0814db-fa3a-11ed-bc8d-00163eee7ae2", "type": "icmp", "ip": "redacted", "id": "sampleid1", "status": "down" }, "field1":"value1", "@version": "1", "agent": { "type": "heartbeat", "ephemeral_id": "a68c31e5-eba7-4944-b88d-d5020917b488", "id": "422d8f5a-bbe0-4a7b-92d9-542ef9192560", "name": "heartbeat_instance", "version": "8.7.0" } }, "fields": { "@timestamp": [ "2023-05-24T13:51:23.557Z" ], "monitor.timespan.lt": [ "2023-05-24T13:51:38.559Z" ], "state.ends.started_at": [ "2023-05-24T13:51:23.559Z" ], "state.started_at": [ "2023-05-24T13:51:28.559Z" ], "monitor.timespan.gte": [ "2023-05-24T13:51:28.559Z" ] } ``` These logs occur one right after another, and should compound the duration that the host has been down, but you can see the state ID changes, and thus all state subfields also reset. Any irrelevant or personal fields have been deleted or redacted. The heartbeat.yml config file has no special settings configured. The output is kafka, and it contains only a couple of processors that copy field values into different field names. ---
**Author:** ![scantron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scantron/32/114018_2.png) [@scantron](https://discuss.elastic.co/u/scantron)\ **Post date:** [June 2, 2023, 3:29pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/7 "2023-06-02T15:29:47Z")
bump ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 30, 2023, 5:30pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/8 "2023-06-30T17:30:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. ---
**Author:** ![emilioalvap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emilioalvap/32/99310_2.png) [@emilioalvap](https://discuss.elastic.co/u/emilioalvap)\ **Post date:** [July 4, 2023, 3:46pm UTC](https://discuss.elastic.co/t/heartbeat-ping-timeout-on-hosts-causes-state-duration-ms-to-stay-at-0/334051/9 "2023-07-04T15:46:49Z")
Hi @scantron, Something that caught my attention about the documents you provided: - In the example \, a new `down` state with `id: default-1884e0759dc-6e52` is ending a previous `up` state with `id: default-1884e074654-6e51`. - Whereas in \, the new `down` state with `id: default-1884e07692f-6cc0` is ending a previous `up` state with `id: default-1884e0755a7-6cbf`. So there're are definitely _two_ distinct `up` states at the time that these monitors ran. I then noticed that both documents have a different agent id, as if there were running on different heartbeat instances: - \ ```auto "agent": { "type": "heartbeat", "ephemeral_id": "4773b85c-53b9-43af-b353-1777ec80b0d9", ``` - \ ```auto "agent": { "type": "heartbeat", "ephemeral_id": "a68c31e5-eba7-4944-b88d-d5020917b488", ``` Can you tell us a bit more about how these are set up? If you're running two, or more, instances of heartbeat with the same monitor config, you'll get a different state id per instance as these are handled locally.