# Trying to fetch status of a web url to determine if its up/down

**URL:** <https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163>\
**Category:** Logstash\
**Created:** [July 6, 2017, 8:00pm UTC](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163 "2017-07-06T20:00:54Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![aman\_yadav](https://avatars.discourse-cdn.com/v4/letter/a/76d3ee/32.png) [@aman\_yadav](https://discuss.elastic.co/u/aman_yadav)\
**Post date:** [July 6, 2017, 8:00pm UTC](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163/1 "2017-07-06T20:00:54Z")

</div>

Hi,

I have a url and I need to find out if this site is up or not. I was trying to do something like below. and then I thought I could get header's statuscode to determine its status.  
But this url returns 'text/html' response, because of which logstash returns an error 'Json parsing error'.

http\_poller {  
urls =\> {  
myurl =\> "[http://myurl.com](http://myurl.com)"  
}  
user =\> "myusername"  
password =\> "mypassword"  
interval =\> 60  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 6, 2017, 8:05pm UTC](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163/2 "2017-07-06T20:05:44Z")

</div>

Have you tried setting `codec => plain` for the http\_poller input?

---

<div class="post-metadata">

**Author:** ![aman\_yadav](https://avatars.discourse-cdn.com/v4/letter/a/76d3ee/32.png) [@aman\_yadav](https://discuss.elastic.co/u/aman_yadav)\
**Post date:** [July 6, 2017, 8:35pm UTC](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163/3 "2017-07-06T20:35:46Z")

</div>

Thanks for the quick reply.  
Now there is no error, but I don't see any statuscode field. Like when we do a http request from node.js or postman. We get statusCode like 200 or 404.  
Is there anything like that in logstash?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 11, 2017, 6:36pm UTC](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163/4 "2017-07-11T18:36:10Z")

</div>

That stuff is by default stored in the `@metadata` field. You're looking for `[@metadata][code]`.

```nohighlight
$ logstash -e 'input { http_poller { urls => { elastic => "http://elastic.co" } interval => 60 codec => plain } } output { stdout { codec => rubydebug { metadata => true } } }'
Settings: Default pipeline workers: 8
Pipeline main started
{
       "message" => "...",
      "@version" => "1",
    "@timestamp" => "2017-07-11T18:34:19.506Z",
     "@metadata" => {
                    "name" => "elastic",
                    "host" => "bertie",
                 "request" => {
            "method" => "get",
               "url" => "http://elastic.co"
        },
         "runtime_seconds" => 0.865,
                    "code" => 200,
        "response_headers" => {
                           "content-type" => "text/html; charset=utf-8",
                      "transfer-encoding" => "chunked",
                             "connection" => "keep-alive",
            "access-control-allow-origin" => "*",
                                   "date" => "Tue, 11 Jul 2017 14:09:09 GMT",
                                   "etag" => "W/\"a18e-uOgaBWwK/0/UiaDoWbH0fw\"",
                                 "server" => "ngx",
                           "x-powered-by" => "Built.io Contentstack",
                              "x-runtime" => "134ms",
                       "x-xss-protection" => "1; mode=block",
                                   "vary" => "Accept-Encoding",
                                    "age" => "15910",
                                "x-cache" => "Hit from cloudfront",
                                    "via" => "1.1 d89eaba973a5c1614ea9f0994f5e8a12.cloudfront.net (CloudFront)",
                            "x-amz-cf-id" => "NA1q3_KShkHrsb389fhGQ-BvSy90I8E199OwCzaSIxEHVI30Virx9A=="
        },
        "response_message" => "OK",
           "times_retried" => 0
    }
}
^CSIGINT received. Shutting down the agent. {:level=>:warn}
stopping pipeline {:id=>"main"}
Pipeline main has been shutdown

```

---

<div class="post-metadata">

**Author:** ![aman\_yadav](https://avatars.discourse-cdn.com/v4/letter/a/76d3ee/32.png) [@aman\_yadav](https://discuss.elastic.co/u/aman_yadav)\
**Post date:** [July 11, 2017, 6:55pm UTC](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163/5 "2017-07-11T18:55:44Z")

</div>

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 8, 2017, 6:55pm UTC](https://discuss.elastic.co/t/trying-to-fetch-status-of-a-web-url-to-determine-if-its-up-down/92163/6 "2017-08-08T18:55:46Z")

</div>

Closed on August 8, 2017, 6:55pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.