Logging events to Logstash via REST with custom request and responses

[dml]

I would like an application to log events to Logstash using a REST API. The request is JSON-encoded event data, some of which are required, some optional, and some would default to default values. The request would need to be processed (perhaps with the help of a filter) and then forwarded to Elasticsearch. When the event is successfully written to Elasticsearch, a JSON-encoded "success" response would be sent back to the application. If any error was encountered during processing (e.g., malformed JSON) or while writing to Elasticsearch (e.g., invalid index provided), a JSON-encoded "failure" response would be sent back to the application.

Is the above use case possible using the existing set of Logstash input, output, and filter plugins? I have read up on logstash-input-http, logstash-output-elasticsearch, and logstash-filter-json. They don't seem to quite do what I am looking for. Should I consider developing custom plugin(s) for this use case? Or put a REST server in between the application and Logstash?

Has anyone done anything similar to this? Any help, suggestions, or examples would be greatly appreciated. Thanks in advance.

[magnusbaeck (Magnus Bäck)]

Is the above use case possible using the existing set of Logstash input, output, and filter plugins?

No, sorry. I think you'll have to write a custom script.

While I obviously don't have the full context, I'd have the application write its log events to disk or submit them to a message broker and assume that the messages will be processed.

[dml]

Thank you for the quick response. Would you recommend creating a web service(s) and writing directly to Elasticsearch, bypassing Logstash altogether?

[magnusbaeck (Magnus Bäck)]

If your application really needs to know whether the log entries were successfully written to Elasticsearch that's probably what you'd have to do, but I challenge that premise. Make your application's logging layer as thin, simple, and non-intrusive as possible (i.e. dump everything as JSON to a file) and let another component worry about getting it into Elasticsearch.