Filebeat proxy permissions?

Elastic Stack Beats
1

I have a nginx proxy restricting access to ES to PUT/POST only, but i am getting Forbidden in filebeat.log. Does filebeat need access to other URL locations?

I am allowing PUT and POST to /filebeat* index and this works:

curl -XPUT 'localhost:8080/filebeat-1.1.2016/test/1?pretty' -d'
{
"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elasticsearch"
}'

but in filebeat logs i find forbidden error and dont get any log on ES.

2016-11-30T18:27:49+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden
2016-11-30T18:27:51+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden
2016-11-30T18:27:55+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden
2016-11-30T18:28:03+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden
2016-11-30T18:28:18+01:00 INFO Non-zero metrics in the last 30s: filebeat.harvester.running=22 filebeat.harvester.started=22 libbeat.es.publish.read_bytes=1620 libbeat.publisher.published_events=1980 filebeat.harvester.open_files=22 libbeat.es.publish.write_bytes=680
2016-11-30T18:28:19+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden
2016-11-30T18:28:48+01:00 INFO Non-zero metrics in the last 30s: filebeat.harvester.open_files=1 libbeat.es.publish.read_bytes=324 libbeat.es.publish.write_bytes=136
2016-11-30T18:28:51+01:00 ERR Connecting error publishing events (retrying): 403 Forbidden

2

Can you turn on some logging on the nginx side to see what the requests are that are failing.

Filebeat 5 does a GET request to check the ES version. It can also check and install index templates depending on your config (GET/PUT to /_template/filebeat).

3

I ran into a similar situation in the past. I had forgotten to also white-list POST requests to the /_bulk API endpoint (or could be /{index}/_bulk if you've set a specific index name)

4

yes, the /_bulk is the problem.

Why is filebeat using /_bulk and not /filebeat-xxx/_bulk? I just want to enable bulk post on /filebeat*

"POST /_bulk HTTP/1.1" 404 169 "-"

I have set index:

output:
elasticsearch:
hosts: ["xxxx:9200"]
index: filebeat

5

I think it will only include the index name in the URI if you specify the index name in the Filebeat config (output.elasticsearch.index: yourindexname-%{+yyyy.MM.dd}). I might be wrong, but @andrewkroh can probably confirm that for us.

6

i have the index name set, but same.

output:
elasticsearch:
hosts: ["xxxx:9200"]
index: filebeat

7

By the way @andrewkroh, can you point me to the specific Go file that builds the URI for the POST request made to Elasticsearch? I'm curious to look a bit further into the low level details of it.

8

@hartfordfive The code is in the libbeat/outputs/elasticsearch package. Here's where the bulk request is being created https://github.com/elastic/beats/blob/master/libbeat/outputs/elasticsearch/bulkapi.go#L63

9

@andrewkroh any idea why filebeat is using /_bulk instead of /filebeat-11.30.2016/_bulk or how to configure it? Thanks

10

Probably because there is support for sending events to different indices. It's easier and more efficient to send a single bulk request and write the various index values in the one bulk request rather than making multiple bulk requests (one for each index).

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.