Logstash Output Elasticsearch bulk_path

Elastic Stack Logstash
1

Maybe I just don't understood the Documentation of bulk_path but as I understood you would use bulk_path to specifiy the index and then logstash automatically appends /_bulk. I noticed running Logstash 6.1.1 and Elasticsearch Output 9.0.2 that you need to explicitly specify /_bulk.

HTTP Path to perform the _bulk requests to this defaults to a concatenation of the path parameter and "_bulk"

output {
  elasticsearch {
			hosts => ["https://elasticsearch:9200/"] 
			bulk_path=> "logstash-preisstaffel/preisstaffel/_bulk"
			}
}

Is this a bug or is it working as intended needing to pass _bulk explicitly?

2

Maybe I just don't understood the Documentation of bulk_path but as I understood you would use bulk_path to specifiy the index and then logstash automatically appends /_bulk.

No, that's not how the documentation should be interpreted. The full path to the bulk endpoint should be specified, including _bulk.

I'd expect it to be extremely rare to have to override this option. Are you sure you need to?

3

I secured my elasticsearch using readonlyrest and the rest.action.multi.allow_explicit_index: false option in my elasticsearch.yml. Using this elasticsearch setting you have to set the bulk_path explicitly via the url and not inside the body of the request. I search the internet how to be able to import data using logstash and a post pointed me to the bulk_path option. It didn't work though. Still got a Errorcode 400.

4

And what's the error message that came with the 400 response?

5

Logstash just outputted that [2018-01-24T12:21:17,433][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>400, :url=>"https://localhost:9200/logstash-gasin-preisstaffel/preisstaffel/_bulk"}.
Same error without the type in the bulk_path.

6

Anything interesting logged on the ES side?

7

elasticsearch was running inside of a container and I did a factory reset of docker after the mobylinuxvm stopped responding. This resulted in the destruction of all logs from that test. I could try recreating it today.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.