# Http output for filebeat?

**URL:** <https://discuss.elastic.co/t/http-output-for-filebeat/62137>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 4, 2016, 9:27am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137 "2016-10-04T09:27:42Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![sunilmchaudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunilmchaudhari/32/9475_2.png) [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\
**Post date:** [October 4, 2016, 9:27am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/1 "2016-10-04T09:27:43Z")

</div>

Hello,  
We have some need where we need to send logs from FB to datapower. We cant send filebeat output directly to ES or logstash.  
Our LS and ES components are inside secure zone. Some clients are outside secure zone, where we will install filebeat and output those logs to logstash via datapower. datapower then just send those logs using pass through url.

Here I am facing problem while configuring FB output to datapower. I need some kind of FB output which will send http message ot datapower and datapower will pass through it to :

Cant we have http output in filebeat llike we have http input to logstash?  
any suggestion???

br,  
Sunil.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [October 4, 2016, 11:28am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/2 "2016-10-04T11:28:25Z")

</div>

The elasticsearch output is based on http. Does this help?

---

<div class="post-metadata">

**Author:** ![sunilmchaudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunilmchaudhari/32/9475_2.png) [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\
**Post date:** [October 4, 2016, 11:28am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/3 "2016-10-04T11:28:58Z")

</div>

Hi,  
I need to try this out...

---

<div class="post-metadata">

**Author:** ![sunilmchaudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunilmchaudhari/32/9475_2.png) [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\
**Post date:** [October 4, 2016, 11:42am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/4 "2016-10-04T11:42:09Z")

</div>

Hi,  
When try to use ES output, getting below error:

```
2016-10-04T17:10:54+05:30 DBG ES Ping(url=http://<DP-host>:8455, timeout=1m30s)
2016-10-04T17:10:54+05:30 DBG Ping request failed with: 405 Method Not Allowed
2016-10-04T17:10:54+05:30 INFO Connecting error publishing events (retrying): 405 Method Not Allowed

```

br,  
Sunil

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 4, 2016, 12:01pm UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/5 "2016-10-04T12:01:20Z")

</div>

which filebeat version are you using? The `ES Ping` used to use `HEAD` requests, unfortunately disallowed by some HTTP proxies. 5.0 will use `GET`.

For generic HTTP see this discussion: [Output beat events as plain HTTP POST](https://discuss.elastic.co/t/output-beat-events-as-plain-http-post/57923)

Code here: [https://github.com/raboof/beats-output-http](https://github.com/raboof/beats-output-http)

You have to compile the beat yourself though + it's not really supported by elastic.

---

<div class="post-metadata">

**Author:** ![sunilmchaudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunilmchaudhari/32/9475_2.png) [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\
**Post date:** [October 19, 2016, 5:02am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/6 "2016-10-19T05:02:41Z")

</div>

Hello,  
I allowed head on the datapower host.  
It now shows error

```
2016-10-18T12:33:17+05:30 DBG Sending bulk request to http://<datapower-host>:8455/_bulk
2016-10-18T12:33:18+05:30 ERR Failed to perform any bulk index operations: invalid character 'o' looking for beginning of value
2016-10-18T12:33:18+05:30 INFO Error publishing events (retrying): invalid character 'o' looking for beginning of value
2016-10-18T12:33:18+05:30 INFO send fail

```

is this something related to the json object being sent?? When I retrieve json object from filebeat debug log and sent to same host:port from browser REST interface then its sent successfully.  
br,  
Sunil

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 19, 2016, 5:19am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/7 "2016-10-19T05:19:47Z")

</div>

The elasticsearch output sends batches of records to Elasticsearch using the [bulk API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html). This is done for performance reasons as sending a single event per request is inefficient. I therefore suspect it may be difficult to get this to work directly with your system.

One way to possibly get around this without adding a custom output to filebeat, could be to have filebeat send data to Logstash and then use the Logstash HTTP output plugin to send data to your system.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 19, 2016, 11:09am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/8 "2016-10-19T11:09:01Z")

</div>

the bulk API response should be a JSON object itself. Parsing seems to fail on the response.

Which exact filebeat version are you using? Can you capture the HTTP request and response using tcpdump?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 25, 2016, 9:27am UTC](https://discuss.elastic.co/t/http-output-for-filebeat/62137/9 "2016-10-25T09:27:46Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
