Trying to fetch status of a web url to determine if its up/down

[aman_yadav]

Hi,

I have a url and I need to find out if this site is up or not. I was trying to do something like below. and then I thought I could get header's statuscode to determine its status.
But this url returns 'text/html' response, because of which logstash returns an error 'Json parsing error'.

http_poller {
urls => {
myurl => "http://myurl.com"
}
user => "myusername"
password => "mypassword"
interval => 60
}

[magnusbaeck (Magnus Bäck)]

Have you tried setting codec => plain for the http_poller input?

[aman_yadav]

Thanks for the quick reply.
Now there is no error, but I don't see any statuscode field. Like when we do a http request from node.js or postman. We get statusCode like 200 or 404.
Is there anything like that in logstash?

[magnusbaeck (Magnus Bäck)]

That stuff is by default stored in the @metadata field. You're looking for [@metadata][code].

$ logstash -e 'input { http_poller { urls => { elastic => "http://elastic.co" } interval => 60 codec => plain } } output { stdout { codec => rubydebug { metadata => true } } }'
Settings: Default pipeline workers: 8
Pipeline main started
{
       "message" => "...",
      "@version" => "1",
    "@timestamp" => "2017-07-11T18:34:19.506Z",
     "@metadata" => {
                    "name" => "elastic",
                    "host" => "bertie",
                 "request" => {
            "method" => "get",
               "url" => "http://elastic.co"
        },
         "runtime_seconds" => 0.865,
                    "code" => 200,
        "response_headers" => {
                           "content-type" => "text/html; charset=utf-8",
                      "transfer-encoding" => "chunked",
                             "connection" => "keep-alive",
            "access-control-allow-origin" => "*",
                                   "date" => "Tue, 11 Jul 2017 14:09:09 GMT",
                                   "etag" => "W/\"a18e-uOgaBWwK/0/UiaDoWbH0fw\"",
                                 "server" => "ngx",
                           "x-powered-by" => "Built.io Contentstack",
                              "x-runtime" => "134ms",
                       "x-xss-protection" => "1; mode=block",
                                   "vary" => "Accept-Encoding",
                                    "age" => "15910",
                                "x-cache" => "Hit from cloudfront",
                                    "via" => "1.1 d89eaba973a5c1614ea9f0994f5e8a12.cloudfront.net (CloudFront)",
                            "x-amz-cf-id" => "NA1q3_KShkHrsb389fhGQ-BvSy90I8E199OwCzaSIxEHVI30Virx9A=="
        },
        "response_message" => "OK",
           "times_retried" => 0
    }
}
^CSIGINT received. Shutting down the agent. {:level=>:warn}
stopping pipeline {:id=>"main"}
Pipeline main has been shutdown

[aman_yadav]

Thanks a lot.