# connecting filebeat with stream API

**URL:** <https://discuss.elastic.co/t/connecting-filebeat-with-stream-api/370513>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 14, 2024, 1:26am UTC](https://discuss.elastic.co/t/connecting-filebeat-with-stream-api/370513 "2024-11-14T01:26:07Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![masber1](https://avatars.discourse-cdn.com/v4/letter/m/919ad9/32.png) [@masber1](https://discuss.elastic.co/u/masber1)\
**Post date:** [November 14, 2024, 1:26am UTC](https://discuss.elastic.co/t/connecting-filebeat-with-stream-api/370513/1 "2024-11-14T01:26:08Z")

</div>

Dear filebeat community,

I am new to elastic and filebeat and need some help.

I am trying to make filebeat to get log events from a hashicorp Nomad server stream api ([Events - HTTP API | Nomad | HashiCorp Developer](https://developer.hashicorp.com/nomad/api-docs/events)) the idea behind is I can hook to this stream api and the Nomad server will send my http client events in json format.

I can replicate this functionality using curl like `curl -s -N --cert /home/user/client.crt --key /home/user/client.key --cacert /home/user/ca.pem --header "X-Nomad-Token: --REDACTED-- https://myserver.local:4646/v1/event/stream?namespace=*`

I am now trying to replicate filebeat so it can receive notifications from this streaming api and print them into stdout.

So far I have not been successful. I tried the http input with the configuration below:

```auto
filebeat.inputs:
- type: http_endpoint
  enabled: true
  listen_address: myserver.local
  listen_port: 4646
  url: "/v1/event/stream?namespace=*"
  secret.header: "X-Nomad-Token"
  secret.value: "--REDACTED--"
  ssl.enabled: true
  ssl.certificate: "/home/user/client.crt"
  ssl.key: "/home/user/client.key"
  ssl.certificate_authority: "/home/user/ca.pem"
  ssl.verification_mode: "full"

output.console:
  pretty: true # Optional, makes the output more readable

```

and then run filebeat like:

```auto
sudo filebeat -e -c ~/filebeat_2.config

```

filebeat starts and I get the following messages:

```auto
{"log.level":"info","@timestamp":"2024-11-14T00:05:38.048+0100","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Start","file.name":"beater/crawler.go","file.line":71},"message":"Loading Inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-11-14T00:05:38.048+0100","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).startInput","file.name":"beater/crawler.go","file.line":117},"message":"starting input, keys present on the config: [filebeat.inputs.0.enabled filebeat.inputs.0.listen_address filebeat.inputs.0.listen_port filebeat.inputs.0.secret.header filebeat.inputs.0.secret.value filebeat.inputs.0.ssl.certificate filebeat.inputs.0.ssl.certificate_authority filebeat.inputs.0.ssl.enabled filebeat.inputs.0.ssl.key filebeat.inputs.0.ssl.verification_mode filebeat.inputs.0.type filebeat.inputs.0.url]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-11-14T00:05:38.057+0100","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).startInput","file.name":"beater/crawler.go","file.line":148},"message":"Starting input (ID: 16785407196084471138)","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-11-14T00:05:38.057+0100","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*crawler).Start","file.name":"beater/crawler.go","file.line":106},"message":"Loading and starting Inputs completed. Enabled inputs: 1","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-11-14T00:05:38.057+0100","log.logger":"input.http_endpoint","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1","file.name":"compat/compat.go","file.line":127},"message":"Input 'http_endpoint' starting","service.name":"filebeat","id":"E8F1BEE974E68562","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-11-14T00:05:38.057+0100","log.logger":"metric_registry","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry","file.name":"inputmon/input.go","file.line":63},"message":"registering","service.name":"filebeat","input_type":"http_endpoint","id":"E8F1BEE974E68562","key":"E8F1BEE974E68562","uuid":"cc216ea1-c1b2-4a0d-895a-30b9e2168784","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-11-14T00:05:38.057+0100","log.logger":"input.http_endpoint","log.origin":{"function":"github.com/elastic/beats/v7/x-pack/filebeat/input/http_endpoint.(*pool).serve","file.name":"http_endpoint/input.go","file.line":212},"message":"Starting HTTPS server on myserver.local:4646 with /v1/event/stream?namespace=* end point","service.name":"filebeat","id":"E8F1BEE974E68562","address":"myserver.local:4646","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-11-14T00:05:38.098+0100","log.logger":"metric_registry","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry.func1","file.name":"inputmon/input.go","file.line":70},"message":"unregistering","service.name":"filebeat","input_type":"http_endpoint","id":"E8F1BEE974E68562","key":"E8F1BEE974E68562","uuid":"cc216ea1-c1b2-4a0d-895a-30b9e2168784","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2024-11-14T00:05:38.098+0100","log.logger":"input.http_endpoint","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1","file.name":"compat/compat.go","file.line":140},"message":"Input 'http_endpoint' failed with: unable to start server due to error: listen tcp 10.0.0.101:4646: bind: cannot assign requested address","service.name":"filebeat","id":"E8F1BEE974E68562","ecs.version":"1.6.0"}

```

I would like to ask for help and if there is a workaround to solve this issue and be able to receive notifications

thank you

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 14, 2024, 2:45am UTC](https://discuss.elastic.co/t/connecting-filebeat-with-stream-api/370513/2 "2024-11-14T02:45:30Z")

</div>

Hello and welcome,

I think there is a little confusion here.

It seems that you want to get logs from Nomad by connecting to an API endpoint, right?

In the example you shared you use curl to talk to an API endpoint.

The `http_endpoint` input is used to make fileabeat **listen** for incoming requests, not to get data from other places.

As the [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-http_endpoint.html) says:

> This input can for example be used to **receive incoming webhooks** from a third-party application or service.

From what I understand you want the opposite, so you need to use the [`httpjson`](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html) input.

But I'm not sure this will work also as the `httpjson` input makes requests in a specific interval and it seems that the endpoint you want to use will send the data as a stream.
