Is there any way to prevent ES from disclosing exception details in REST response?

We expose ES _search endpoint directly to consumers. When our REST API get
scanned for security vulnerabilities it complains on ES returning exception
details. For example a malformed query will be included in the response
along with exception. While it is more or a less harmless the tool
complains of various injections and internals disclosures. I would like to
be able to turn error message in the response off (or substitute it with a
generic message) in production while keeping normal response logic in
development.

Is there any way I can do it?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You can put a revers proxy like nginx between ES cluster and the rest of
the world and filter away all HTTP status 500 responses.

Jörg

On Mon, Sep 15, 2014 at 11:57 PM, AlexR roytmana@gmail.com wrote:

We expose ES _search endpoint directly to consumers. When our REST API get
scanned for security vulnerabilities it complains on ES returning exception
details. For example a malformed query will be included in the response
along with exception. While it is more or a less harmless the tool
complains of various injections and internals disclosures. I would like to
be able to turn error message in the response off (or substitute it with a
generic message) in production while keeping normal response logic in
development.

Is there any way I can do it?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Jorg,

Unfortunately it is not an option - we are not at liberty to touch anything
beyond our app servers. We are using transport-wares servlet for ES and I
could easily tweak AbstractServletRestChannel to handle Rest Channel
response with codes 400,500 but I would like to avoid modifying the code
directly and there is no way to do it nicely. I put a request on github for
enhancements of the NodeServlet but was hoping ES may have an option to
turn error details on/off. I think it would be nice to control error level
in REST responses with three levels - suppress/message/stack-trace

On Mon, Sep 15, 2014 at 6:01 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can put a revers proxy like nginx between ES cluster and the rest of
the world and filter away all HTTP status 500 responses.

Jörg

On Mon, Sep 15, 2014 at 11:57 PM, AlexR roytmana@gmail.com wrote:

We expose ES _search endpoint directly to consumers. When our REST API
get scanned for security vulnerabilities it complains on ES returning
exception details. For example a malformed query will be included in the
response along with exception. While it is more or a less harmless the tool
complains of various injections and internals disclosures. I would like to
be able to turn error message in the response off (or substitute it with a
generic message) in production while keeping normal response logic in
development.

Is there any way I can do it?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Qx1TPSMS9ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAY7rMRi4zsnv7QapKEr3iwyWxY0EDOnDczDeYa_0nzsFYT9sQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Then why don't you simply add a servlet filter that filters unwanted
responses away?

Jörg

On Tue, Sep 16, 2014 at 12:21 AM, Alex Roytman roytmana@gmail.com wrote:

Thanks Jorg,

Unfortunately it is not an option - we are not at liberty to touch
anything beyond our app servers. We are using transport-wares servlet for
ES and I could easily tweak AbstractServletRestChannel to handle Rest
Channel response with codes 400,500 but I would like to avoid modifying the
code directly and there is no way to do it nicely. I put a request on
github for enhancements of the NodeServlet but was hoping ES may have an
option to turn error details on/off. I think it would be nice to control
error level in REST responses with three levels -
suppress/message/stack-trace

On Mon, Sep 15, 2014 at 6:01 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can put a revers proxy like nginx between ES cluster and the rest of
the world and filter away all HTTP status 500 responses.

Jörg

On Mon, Sep 15, 2014 at 11:57 PM, AlexR roytmana@gmail.com wrote:

We expose ES _search endpoint directly to consumers. When our REST API
get scanned for security vulnerabilities it complains on ES returning
exception details. For example a malformed query will be included in the
response along with exception. While it is more or a less harmless the tool
complains of various injections and internals disclosures. I would like to
be able to turn error message in the response off (or substitute it with a
generic message) in production while keeping normal response logic in
development.

Is there any way I can do it?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Qx1TPSMS9ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAY7rMRi4zsnv7QapKEr3iwyWxY0EDOnDczDeYa_0nzsFYT9sQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAAY7rMRi4zsnv7QapKEr3iwyWxY0EDOnDczDeYa_0nzsFYT9sQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH8fWhWpAKU4MT6oVxgRNG%2BBS1Y3QdOyn2eWiqwCErsJQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I guess I could but it would mean passing a response wrapper to capture
output stream and then copy it to real request or discard it in case of an
error. That would be a second copy of response - the first one being done
in the NodeServlet - will hurt performance for large responses :frowning:

On Mon, Sep 15, 2014 at 6:40 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Then why don't you simply add a servlet filter that filters unwanted
responses away?

Jörg

On Tue, Sep 16, 2014 at 12:21 AM, Alex Roytman roytmana@gmail.com wrote:

Thanks Jorg,

Unfortunately it is not an option - we are not at liberty to touch
anything beyond our app servers. We are using transport-wares servlet for
ES and I could easily tweak AbstractServletRestChannel to handle Rest
Channel response with codes 400,500 but I would like to avoid modifying the
code directly and there is no way to do it nicely. I put a request on
github for enhancements of the NodeServlet but was hoping ES may have an
option to turn error details on/off. I think it would be nice to control
error level in REST responses with three levels -
suppress/message/stack-trace

On Mon, Sep 15, 2014 at 6:01 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can put a revers proxy like nginx between ES cluster and the rest of
the world and filter away all HTTP status 500 responses.

Jörg

On Mon, Sep 15, 2014 at 11:57 PM, AlexR roytmana@gmail.com wrote:

We expose ES _search endpoint directly to consumers. When our REST API
get scanned for security vulnerabilities it complains on ES returning
exception details. For example a malformed query will be included in the
response along with exception. While it is more or a less harmless the tool
complains of various injections and internals disclosures. I would like to
be able to turn error message in the response off (or substitute it with a
generic message) in production while keeping normal response logic in
development.

Is there any way I can do it?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Qx1TPSMS9ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAY7rMRi4zsnv7QapKEr3iwyWxY0EDOnDczDeYa_0nzsFYT9sQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAAY7rMRi4zsnv7QapKEr3iwyWxY0EDOnDczDeYa_0nzsFYT9sQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Qx1TPSMS9ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH8fWhWpAKU4MT6oVxgRNG%2BBS1Y3QdOyn2eWiqwCErsJQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH8fWhWpAKU4MT6oVxgRNG%2BBS1Y3QdOyn2eWiqwCErsJQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAAY7rMTDugDvhOOHnxe1PXqPLYNNa_GbpQEjeaXFwa3A%3DnFrtw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

You could just check for the response code 500, and you're done, no need to
capture streams.

Jörg

On Tue, Sep 16, 2014 at 12:53 AM, Alex Roytman roytmana@gmail.com wrote:

I guess I could but it would mean passing a response wrapper to capture
output stream and then copy it to real request or discard it in case of an
error. That would be a second copy of response - the first one being done
in the NodeServlet - will hurt performance for large responses :frowning:

On Mon, Sep 15, 2014 at 6:40 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Then why don't you simply add a servlet filter that filters unwanted
responses away?

Jörg

On Tue, Sep 16, 2014 at 12:21 AM, Alex Roytman roytmana@gmail.com
wrote:

Thanks Jorg,

Unfortunately it is not an option - we are not at liberty to touch
anything beyond our app servers. We are using transport-wares servlet for
ES and I could easily tweak AbstractServletRestChannel to handle Rest
Channel response with codes 400,500 but I would like to avoid modifying the
code directly and there is no way to do it nicely. I put a request on
github for enhancements of the NodeServlet but was hoping ES may have an
option to turn error details on/off. I think it would be nice to control
error level in REST responses with three levels -
suppress/message/stack-trace

On Mon, Sep 15, 2014 at 6:01 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You can put a revers proxy like nginx between ES cluster and the rest
of the world and filter away all HTTP status 500 responses.

Jörg

On Mon, Sep 15, 2014 at 11:57 PM, AlexR roytmana@gmail.com wrote:

We expose ES _search endpoint directly to consumers. When our REST API
get scanned for security vulnerabilities it complains on ES returning
exception details. For example a malformed query will be included in the
response along with exception. While it is more or a less harmless the tool
complains of various injections and internals disclosures. I would like to
be able to turn error message in the response off (or substitute it with a
generic message) in production while keeping normal response logic in
development.

Is there any way I can do it?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/83e6b126-4db1-44cc-9a0a-0dd6c6d44a64%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Qx1TPSMS9ro/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHdM-ih%2BW_XCgHiphemWSSGtmK2Z5C%3D%2BiWD-ciBokbWHg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAY7rMRi4zsnv7QapKEr3iwyWxY0EDOnDczDeYa_0nzsFYT9sQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAAY7rMRi4zsnv7QapKEr3iwyWxY0EDOnDczDeYa_0nzsFYT9sQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Qx1TPSMS9ro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH8fWhWpAKU4MT6oVxgRNG%2BBS1Y3QdOyn2eWiqwCErsJQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH8fWhWpAKU4MT6oVxgRNG%2BBS1Y3QdOyn2eWiqwCErsJQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAAY7rMTDugDvhOOHnxe1PXqPLYNNa_GbpQEjeaXFwa3A%3DnFrtw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAAY7rMTDugDvhOOHnxe1PXqPLYNNa_GbpQEjeaXFwa3A%3DnFrtw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHF8PiP%3DSR7hsTYRpOdHvKfoQBMC2feUCHdCgs2ExEANA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I may be missing something but in the filter the check for error will need
to be done after calling chain.doFilter(req, resp); (or we would not know
the status which is set by the NodeServlet). At that point it is too late
to do anything about response body if the output stream was written to.
That's the reason for capturing stream in memory by using response wrapper
and then writing or not writing it depending on status to the real response
object

On Tuesday, September 16, 2014 3:10:25 AM UTC-4, Jörg Prante wrote:

You could just check for the response code 500, and you're done, no need
to capture streams.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6da2ac62-ead6-4588-aea6-a69e330e9e04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.