# Filebeat proxy permissions?

**URL:** https://discuss.elastic.co/t/filebeat-proxy-permissions/67658
**Category:** Beats
**Tags:** filebeat
**Created:** [November 30, 2016, 5:38pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658 "2016-11-30T17:38:39Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![agonzalez](https://avatars.discourse-cdn.com/v4/letter/a/ba8739/32.png) [@agonzalez](https://discuss.elastic.co/u/agonzalez)
#### Post date: [November 30, 2016, 5:38pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/1 "2016-11-30T17:38:39Z")

</div>

I have a nginx proxy restricting access to ES to PUT/POST only, but i am getting Forbidden in filebeat.log. Does filebeat need access to other URL locations?

I am allowing PUT and POST to /filebeat\* index and this works:

curl -XPUT 'localhost:8080/filebeat-1.1.2016/test/1?pretty' -d'  
{  
"user" : "kimchy",  
"post\_date" : "2009-11-15T14:12:12",  
"message" : "trying out Elasticsearch"  
}'

but in filebeat logs i find forbidden error and dont get any log on ES.

2016-11-30T18:27:49+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden  
2016-11-30T18:27:51+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden  
2016-11-30T18:27:55+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden  
2016-11-30T18:28:03+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden  
2016-11-30T18:28:18+01:00 INFO Non-zero metrics in the last 30s: filebeat.harvester.running=22 filebeat.harvester.started=22 libbeat.es.publish.read\_bytes=1620 libbeat.publisher.published\_events=1980 filebeat.harvester.open\_files=22 libbeat.es.publish.write\_bytes=680  
2016-11-30T18:28:19+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden  
2016-11-30T18:28:48+01:00 INFO Non-zero metrics in the last 30s: filebeat.harvester.open\_files=1 libbeat.es.publish.read\_bytes=324 libbeat.es.publish.write\_bytes=136  
2016-11-30T18:28:51+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [November 30, 2016, 5:51pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/2 "2016-11-30T17:51:59Z")

</div>

Can you turn on some logging on the nginx side to see what the requests are that are failing.

Filebeat 5 does a GET request to check the ES version. It can also check and install index templates depending on your config (GET/PUT to /\_template/filebeat).

---

<div class="post-metadata">

### Author: ![hartfordfive](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hartfordfive/32/44794_2.png) [@hartfordfive](https://discuss.elastic.co/u/hartfordfive)
#### Post date: [November 30, 2016, 5:57pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/3 "2016-11-30T17:57:33Z")

</div>

I ran into a similar situation in the past. I had forgotten to also white-list POST requests to the `/_bulk` API endpoint (or could be `/{index}/_bulk` if you've set a specific index name)

---

<div class="post-metadata">

### Author: ![agonzalez](https://avatars.discourse-cdn.com/v4/letter/a/ba8739/32.png) [@agonzalez](https://discuss.elastic.co/u/agonzalez)
#### Post date: [November 30, 2016, 6:10pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/4 "2016-11-30T18:10:57Z")

</div>

yes, the /\_bulk is the problem.

Why is filebeat using /\_bulk and not /filebeat-xxx/\_bulk? I just want to enable bulk post on /filebeat\*

"POST /\_bulk HTTP/1.1" 404 169 "-"

I have set index:

output:  
elasticsearch:  
hosts: ["xxxx:9200"]  
index: filebeat

---

<div class="post-metadata">

### Author: ![hartfordfive](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hartfordfive/32/44794_2.png) [@hartfordfive](https://discuss.elastic.co/u/hartfordfive)
#### Post date: [November 30, 2016, 6:13pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/5 "2016-11-30T18:13:18Z")

</div>

I think it will only include the index name in the URI if you specify the index name in the Filebeat config (`output.elasticsearch.index: yourindexname-%{+yyyy.MM.dd}`). I might be wrong, but @andrewkroh can probably confirm that for us.

---

<div class="post-metadata">

### Author: ![agonzalez](https://avatars.discourse-cdn.com/v4/letter/a/ba8739/32.png) [@agonzalez](https://discuss.elastic.co/u/agonzalez)
#### Post date: [November 30, 2016, 6:14pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/6 "2016-11-30T18:14:30Z")

</div>

i have the index name set, but same.

output:  
elasticsearch:  
hosts: ["xxxx:9200"]  
index: filebeat

---

<div class="post-metadata">

### Author: ![hartfordfive](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hartfordfive/32/44794_2.png) [@hartfordfive](https://discuss.elastic.co/u/hartfordfive)
#### Post date: [November 30, 2016, 6:16pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/7 "2016-11-30T18:16:49Z")

</div>

By the way @andrewkroh, can you point me to the specific Go file that builds the URI for the POST request made to Elasticsearch? I'm curious to look a bit further into the low level details of it.

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [November 30, 2016, 6:36pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/8 "2016-11-30T18:36:45Z")

</div>

@hartfordfive The code is in the libbeat/outputs/elasticsearch package. Here's where the bulk request is being created [https://github.com/elastic/beats/blob/master/libbeat/outputs/elasticsearch/bulkapi.go#L63](https://github.com/elastic/beats/blob/master/libbeat/outputs/elasticsearch/bulkapi.go#L63)

---

<div class="post-metadata">

### Author: ![agonzalez](https://avatars.discourse-cdn.com/v4/letter/a/ba8739/32.png) [@agonzalez](https://discuss.elastic.co/u/agonzalez)
#### Post date: [November 30, 2016, 6:41pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/9 "2016-11-30T18:41:06Z")

</div>

@andrewkroh any idea why filebeat is using /\_bulk instead of /filebeat-11.30.2016/\_bulk or how to configure it? Thanks

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [November 30, 2016, 6:48pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/10 "2016-11-30T18:48:47Z")

</div>

Probably because there is support for sending events to [different indices](https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#_indices). It's easier and more efficient to send a single bulk request and write the various index values in the one bulk request rather than making multiple bulk requests (one for each index).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 28, 2016, 6:48pm UTC](https://discuss.elastic.co/t/filebeat-proxy-permissions/67658/11 "2016-12-28T18:48:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
