Http_endpoint only accepts JSON, even with a different content_type

Elastic Stack Beats
1

We would like to use filebeat to accept data from Apache Nifi in the form of a POST request, run it through some processors and send it to Elasticsearch. The documentation on the http_endpoint seems to suggest that the http_endpoint usually expects data to be JSON-formatted - however this can be overruled by setting the content_type setting to something else:

By default the input expects the incoming POST to include a Content-Type of application/json to try to enforce the incoming data to be valid JSON. In certain scenarios when the source of the request is not able to do that, it can be overwritten with another value or set to null.

This is required for us because we receive large volumes of CSV data. However, it seems like the http_endpoint expects data to always be JSON-formatted, no matter what the content_type is configured to be.

Example filebeat.yml:

filebeat.inputs:
- type: http_endpoint
  enabled: true
  listen_address: 0.0.0.0
  listen_port: 8889
  content_type: ""

output.console:

Example curl command once you start filebeat:

$ curl -d 'this,is,a,test' localhost:8889/ -XPOST
{"message": "Only JSON objects are accepted"}

Am I simply misunderstanding the documentation and this is intended functionality, or could this be a bug?

Version details:
$ ./filebeat version
filebeat version 7.13.2 (amd64), libbeat 7.13.2 [686ba416a74193f2e69dcfa2eb142f4364a79307 built 2021-06-10 21:04:13 +0000 UTC]

2

Hi!

content_type is meant only for content checks, you can leave it blank if the sender does not add the appropriate header but the content still needs to be a valid json.

3

Thanks for the clarification. IMO it would be helpful if the documentation on this input explicitly explained that only JSON is accepted - as it is written, it can confuse noobs such as myself :slight_smile:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.