# Http\_poller -\> graphite

**URL:** <https://discuss.elastic.co/t/http-poller-graphite/26736>\
**Category:** Logstash\
**Created:** [August 3, 2015, 4:49pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736 "2015-08-03T16:49:17Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 3, 2015, 4:49pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/1 "2015-08-03T16:49:17Z")

</div>

Trying to use the new input-http\_poller. I've got an application which on a url, spits out json which looks like this in stdout/rubydebugger:

```auto
{
  "metrics" => {
     "requests" => 12,
     "foo" => 2345,
     "bar" => 999
  }
}

```

Is there an easy way to say in a graphite output block: "Take everything under 'metrics', and send to graphite?

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 3, 2015, 5:57pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/2 "2015-08-03T17:57:43Z")

</div>

Digging deeper into the graphite plugin, it looks like something like this should work:

```auto
if [http_poller_metadata] {
    graphite {
      fields_are_metrics => true
      include_metrics => [".*metrics.*"]
      metrics_format => "poller.%{host}.*"
    }
  }

```

But it doesn't appear to be sending anything.

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 3, 2015, 6:09pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/3 "2015-08-03T18:09:51Z")

</div>

Doing some debugging with:

```auto
stdout {
      codec => graphite {
        fields_are_metrics => true
        metrics_format => "poller.%{host}.*"
      }
    }

```

It appears that it doesn't deal with nested fields at all. All it displays output for is the top-level field. Nothing deeper.

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 3, 2015, 9:26pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/4 "2015-08-03T21:26:31Z")

</div>

So I think I can make it work if I do two things:

1. Just drop all fields except for the nested metric field.
2. Iterate over the metric field, and create new fields/values based on them.

Does anyone have ruby code that can do this in a filter, if I assume the 'metric' field is never more than 1 level deep?

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [August 5, 2015, 3:11pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/5 "2015-08-05T15:11:13Z")

</div>

Hey Bruce, I'll take a look at this later today!

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 5, 2015, 4:39pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/6 "2015-08-05T16:39:45Z")

</div>

Thanks!

I think this sort of thing could make http-poller super useful, in conjunction with graphite.

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 5, 2015, 4:44pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/7 "2015-08-05T16:44:27Z")

</div>

I also created this enhancement request for the graphite output plugin:

> <https://github.com/logstash-plugins/logstash-output-graphite/issues/13>

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [August 5, 2015, 5:55pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/8 "2015-08-05T17:55:39Z")

</div>

Thanks Bruce. I agree that the root cause is missing functionality in the graphite output. I'll try and put a patch in today.

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [August 6, 2015, 5:05pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/9 "2015-08-06T17:05:43Z")

</div>

This has been fixed in the 1.0.1 release of the graphite output. Thanks bringing up the issue guys!

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 6, 2015, 5:20pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/10 "2015-08-06T17:20:19Z")

</div>

Thanks for the quick turn around! I'm trying to use it right now.

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 6, 2015, 5:34pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/11 "2015-08-06T17:34:46Z")

</div>

Not working yet. Here's what I see in logstash.stdout from http-poller. (Trimming out other stuff.)

```auto
{
                "MetricsC" => {
        "requests" => 3
    },
}

```

Here's my config:

```auto
input {
  http_poller {

    urls => {
      test => "https://test.com/status"
    }
    request_timeout => 30
    interval => 300
    codec => "json"
    metadata_target => "http_poller_metadata"
  }
}
filter {
  if [http_poller_metadata] {
      mutate {
        add_field => {
          "host" => "%{http_poller_metadata[name]}"
        }
      }
    }
}
output {
  if [http_poller_metadata] {
    stdout {
      codec => graphite {
        fields_are_metrics => true
        include_metrics => ["MetricsC"]
        metrics_format => "test.infra.poller.%{host}.*"
      }
    }
    graphite {
      host => "xxxxxx"
      fields_are_metrics => true
      include_metrics => ["MetricsC"]
      metrics_format => "test.infra.poller.%{host}.*"
    }
  }
  stdout {
    codec => rubydebug
  }
}

```

Am I doing anything wrong?

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 6, 2015, 6:03pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/12 "2015-08-06T18:03:12Z")

</div>

I think I got it to work. Didn't seem to like my %{host} in metrics format. Testing more.

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 6, 2015, 7:27pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/13 "2015-08-06T19:27:48Z")

</div>

Okay, If I do include\_metrics =\> ["MetricsC"], that works properly.

However, I want to also get http\_poller\_metadata.runtime\_seconds and http\_poller\_metadata.code.

So I tried this:

```auto
      include_metrics => ["MetricsC","http_poller_metadata"]

```

That works for getting everything.

How would I restrict that to just code and runtime\_seconds?

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 6, 2015, 7:40pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/14 "2015-08-06T19:40:06Z")

</div>

Also, it appears that variables aren't interpolated inside of 'metrics\_format':

```auto
      metrics_format => "team.infra.poller.%{host}.*"

```

That results in the directory %{host} in graphite.

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 6, 2015, 7:46pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/15 "2015-08-06T19:46:41Z")

</div>

> [@Bruce\_Lysik](#):
>
> etrics\_format =\> "team.infra.poller.%{host}.\*"

> <https://github.com/logstash-plugins/logstash-output-graphite/issues/15>
>
> \`\`\`
> metrics\_format =\> "team.infra.poller.%{host}.\*"
> \`\`\`
> 
> Results in a actual dir…ectory on the graphite filesystem of %{host}. What's expected is the value of that field.

---

<div class="post-metadata">

**Author:** ![Bruce\_Lysik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_lysik/32/2037_2.png) [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\
**Post date:** [August 6, 2015, 11:06pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/16 "2015-08-06T23:06:27Z")

</div>

For people following along, Andrew added that enhancement, and now metrics\_format works as expected.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:32am UTC](https://discuss.elastic.co/t/http-poller-graphite/26736/17 "2017-07-06T05:32:41Z")

</div>