# Http requests as a beat

**URL:** <https://discuss.elastic.co/t/http-requests-as-a-beat/148019>\
**Category:** Beats\
**Created:** [September 10, 2018, 9:03pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019 "2018-09-10T21:03:41Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![mattz](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mattz](https://discuss.elastic.co/u/mattz)\
**Post date:** [September 10, 2018, 9:03pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/1 "2018-09-10T21:03:41Z")

</div>

I have a small golang application that listens to http post requests on various routes. The data received represents messages on a messaging bus. Our current solution is to write json to a file that's tail'd by filebeat.

I would like to send the json encoded messages to logstash or elasticsearch directly.

Is something like this achievable with libbeat?

Any advice for a new beats developer?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 10, 2018, 9:10pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/2 "2018-09-10T21:10:13Z")

</div>

[https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-http.html](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-http.html) might do what you want?

---

<div class="post-metadata">

**Author:** ![mattz](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mattz](https://discuss.elastic.co/u/mattz)\
**Post date:** [September 10, 2018, 9:15pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/3 "2018-09-10T21:15:06Z")

</div>

I don't think so.

These are requests _to_ my application. The http module is for making outbound requests.

But maybe missing something. I'm here obviously.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 10, 2018, 9:17pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/4 "2018-09-10T21:17:43Z")

</div>

You can call your application as long as it has a HTTP API;

> The HTTP module is a Metricbeat module **used to call arbitrary HTTP endpoints** for which a dedicated Metricbeat module is not available.

---

<div class="post-metadata">

**Author:** ![mattz](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mattz](https://discuss.elastic.co/u/mattz)\
**Post date:** [September 10, 2018, 10:58pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/5 "2018-09-10T22:58:03Z")

</div>

> [@warkolm](#):
>
> You can call your application as long as it has a HTTP API;

I'm not trying to call an API. I'm trying to _listen_ to requests coming in to a web server.

I would like the route handlers to write directly to elasticsearch or logstash using libbeat.

The http module seems like a "curlbeat". Polling an API resource. I don't want to poll a resource.

It's a push mechanism. Not a poll.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 10, 2018, 10:59pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/6 "2018-09-10T22:59:42Z")

</div>

Would [https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-http-options.html](https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-http-options.html) do it then?

---

<div class="post-metadata">

**Author:** ![mattz](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mattz](https://discuss.elastic.co/u/mattz)\
**Post date:** [September 10, 2018, 11:03pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/7 "2018-09-10T23:03:35Z")

</div>

Packetbeat. Interesting thought. Didn't think about that.

Can custom modules be written for packetbeat?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 10, 2018, 11:04pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/8 "2018-09-10T23:04:21Z")

</div>

Do you mean protocols, or modules like filebeat ones?

---

<div class="post-metadata">

**Author:** ![mattz](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mattz](https://discuss.elastic.co/u/mattz)\
**Post date:** [September 11, 2018, 12:07am UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/9 "2018-09-11T00:07:49Z")

</div>

> [@warkolm](#):
>
> Do you mean protocols, or modules like filebeat ones?

I guess so.

I'll read up on packetbeat.

---

<div class="post-metadata">

**Author:** ![mattz](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mattz](https://discuss.elastic.co/u/mattz)\
**Post date:** [September 13, 2018, 7:02pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/10 "2018-09-13T19:02:26Z")

</div>

@warkolm I'm not sure packetbeat solves what we are trying to achieve at this time. It could work (and be really cool) if we were to write a protocol or configure http protocol to parse Multipart/Mixed content type and ship the results.

I found [GitHub - martinhoefling/saltbeat: Saltbeat is a forwarder for messages from the salt master event bus to logstash or elasticsearch](https://github.com/martinhoefling/saltbeat) which seems similar in concept.

> Saltbeat is a forwarder for messages from the salt master event bus to logstash or elasticsearch

I'm going to check out that project.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 11, 2018, 9:02pm UTC](https://discuss.elastic.co/t/http-requests-as-a-beat/148019/11 "2018-10-11T21:02:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
