# Inconsistent response code from heartbeat and curl command from same machine

**URL:** <https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793>\
**Category:** Beats\
**Tags:** heartbeat\
**Created:** [December 13, 2019, 12:52pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793 "2019-12-13T12:52:27Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![J\_Weeda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/j_weeda/32/45092_2.png) [@J\_Weeda](https://discuss.elastic.co/u/J_Weeda)\
**Post date:** [December 13, 2019, 12:52pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/1 "2019-12-13T12:52:27Z")

</div>

Hi, I'm facing an issue where heartbeat 7.5 reports a different response code (500) value than cUrl (302) reports on the same target URL I believe the heartbeat can me mimicked through `curl -Iv`.

Below the heartbeat configuration in monitor.d, the corresponding response doc and curl output. What could be causing this? BTW, I have tried both with and without a trailing slash in `urls`.

```
- type: http
  name: 'somesite.com'
  tags: ['customer']
  id: 'somesite.com'
  schedule: '@every 60s'
  urls: ["http://somesite.com:80/"]
  timeout: 5
  check.request.method: HEAD

{
  "_index": "heartbeat-7.5.0-customer-2019.12",
  "_type": "_doc",
  "_id": "4ThJ_24B-hakpvoIg4XP",
  "_version": 1,
  "_score": null,
  "_size": 1479,
  "_source": {
    "observer": {
      "ip": "###########",
      "geo": {
        "name": "AWS EU Frankfurt",
        "country_iso_code": "DE",
        "city_name": "Frankfurt",
        "continent_name": "Europe"
      },
      "hostname": "###########"
    },
    "summary": {
      "up": 0,
      "down": 1
    },
    "fields": {
      "customer": "customer",
      "logstash_used": "true",
      "index": "heartbeat-7.5.0-customer-2019.12",
      "logstash_host": "ip-172-31-24-135",
      "size": 1604
    },
    "tags": [
      "customer",
      "beats_input_raw_event"
    ],
    "host": {
      "name": "###########"
    },
    "resolve": {
      "ip": "###########",
      "rtt": {
        "us": 321170
      }
    },
    "@version": "1",
    "@timestamp": "2019-12-13T12:43:47.074Z",
    "url": {
      "scheme": "http",
      "full": "http://somesite.com:80/",
      "domain": "somesite.com",
      "path": "/",
      "port": 80
    },
    "error": {
      "type": "validate",
      "message": "500 Internal Server Error"
    },
    "ecs": {
      "version": "1.1.0"
    },
    "monitor": {
      "type": "http",
      "ip": "###########",
      "check_group": "10eedc6e-1da6-11ea-8856-029326932cd8",
      "name": "somesite.com",
      "duration": {
        "us": 357687
      },
      "id": "somesite.com",
      "status": "down"
    },
    "tcp": {
      "rtt": {
        "connect": {
          "us": 15703
        }
      }
    },
    "event": {
      "dataset": "uptime"
    },
    "agent": {
      "type": "heartbeat",
      "hostname": "###########",
      "id": "28dd306a-1424-42ce-a543-d68e85641532",
      "ephemeral_id": "cd79aee3-8de9-471d-8616-9cc991a3bbbd",
      "version": "7.5.0"
    },
    "http": {
      "response": {
        "body": {
          "bytes": 0,
          "hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
          "content": ""
        },
        "status_code": 500
      },
      "rtt": {
        "total": {
          "us": 36474
        },
        "content": {
          "us": 31
        },
        "response_header": {
          "us": 20692
        },
        "write_request": {
          "us": 25
        },
        "validate": {
          "us": 20723
        }
      }
    }
  },
  "fields": {
    "availability": [
      0
    ],
    "url.link": [
      "http://somesite.com"
    ],
    "@timestamp": [
      "2019-12-13T12:43:47.074Z"
    ],
    "url.suffix": [
      "au"
    ]
  },
  "highlight": {
    "monitor.name": [
      "@kibana-highlighted-field@somesite.com@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1576241027074
  ]
}

```

curl command:

```
curl -Iv http://somesite.com:80
* Rebuilt URL to: http://somesite.com:80/
* Trying ###########...
* TCP_NODELAY set
* Connected to somesite.com (###########) port 80 (#0)
> GET / HTTP/1.1
> Host: somesite.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Keep-Alive: timeout=5, max=1024
< Content-Type: text/html; charset=UTF-8
< Location: http://somesite.com/studio/agegate?rurl=/
< Server: Apache
< Date: Fri, 13 Dec 2019 12:39:21 GMT
< Content-Length: 0
<
* Connection #0 to host somesite.com left intact
```

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [December 16, 2019, 2:02pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/2 "2019-12-16T14:02:02Z")

</div>

Hi @J_Weeda,

I think this can be an issue with redirect support, it is planned to improve it in 7.6.0 after this change: [https://github.com/elastic/beats/pull/14125](https://github.com/elastic/beats/pull/14125)

In the meantime you can try to disable redirections by setting `max_redirects` option to 0 (this will be the default in 7.6.0).

If disabling redirections doesn't fix the issue it'd be great if you could try the patch previously mentioned.

Thanks!

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [December 16, 2019, 3:59pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/3 "2019-12-16T15:59:57Z")

</div>

This could potentially also be the result of a different user agent.

It'd be helpful to dive into the server-side of the 500 error and see what's going on. If a server is returning a 500 that generally indicates a bug on the server side.

---

<div class="post-metadata">

**Author:** ![J\_Weeda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/j_weeda/32/45092_2.png) [@J\_Weeda](https://discuss.elastic.co/u/J_Weeda)\
**Post date:** [December 16, 2019, 5:57pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/4 "2019-12-16T17:57:04Z")

</div>

thanks for your replies...

```
- type: http
  name: 'somesite.com'
  tags: ['customer']
  id: 'somesite.com'
  schedule: '@every 60s'
  urls: ["http://somesite.com:80"]
  max_redirects: 0
  timeout: 5
  check.request.method: HEAD
  headers:
    'Host': 'somesite.com'
    'User-Agent': 'curl/7.58.0'

```

I have made above changes but still the same result. The headers are the same as the cUrl request uses. Is `max_redirects: 0` supposed to work in 7.5 already or have I misread your suggestion @jsoriano?

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [December 16, 2019, 6:26pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/5 "2019-12-16T18:26:47Z")

</div>

`max_redirects: 0` should work correctly in 7.5.

Can you view the logs of the server generating the 500 error? That should have more info as to the precise cause of issues here.

---

<div class="post-metadata">

**Author:** ![J\_Weeda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/j_weeda/32/45092_2.png) [@J\_Weeda](https://discuss.elastic.co/u/J_Weeda)\
**Post date:** [December 16, 2019, 6:29pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/6 "2019-12-16T18:29:42Z")

</div>

@Andrew_Cholakian1, I will try but their not under my control.

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [December 16, 2019, 6:35pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/7 "2019-12-16T18:35:03Z")

</div>

@J_Weeda one question that may help here: why are you manually setting the `Host` header in your request?

---

<div class="post-metadata">

**Author:** ![J\_Weeda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/j_weeda/32/45092_2.png) [@J\_Weeda](https://discuss.elastic.co/u/J_Weeda)\
**Post date:** [December 16, 2019, 6:43pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/8 "2019-12-16T18:43:15Z")

</div>

I wasn't initially but I wanted to explain why the cUrl request works and heartbeat didn't. These two headers are added by cUrl. I started with the `user-agent` and then also added `host` since I didn't notice any difference.

But there is some success to report back. The config I posted (in which the headers where included) was not correct. It should have been like this:

```
- type: http
  name: 'somesite.com'
  tags: ['customer']
  id: 'somesite.com'
  schedule: '@every 60s'
  urls: ["http://somesite.com:80"]
  max_redirects: 0
  timeout: 5
  check.request:
    method: HEAD
    headers:
      'Host': 'somesite.com'
      'User-Agent': 'curl/7.58.0'

```

in fact the config worked without the `Host` header as well. The site reported a 500 because the `User-Agent` header was missing (or not accepted).

@Andrew_Cholakian1, @jsoriano Thanks for your support!

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [December 16, 2019, 7:34pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/9 "2019-12-16T19:34:50Z")

</div>

Ah, makes sense. Thanks for letting us know it's been resolved!

So, to summarize the issue was that the service returned a 500 for the heartbeat default user agent (which is the go user agent). This should (probably?) cease to be an issue in 7.6.0 with the new default agent. It really depends on how the service on the other side behaves.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 13, 2020, 7:34pm UTC](https://discuss.elastic.co/t/inconsistent-response-code-from-heartbeat-and-curl-command-from-same-machine/211793/10 "2020-01-13T19:34:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.