Is there any way of tracking request id?

Elastic Stack Elasticsearch
1

Dear community,

I am looking for a feature which allows to attach requestId to the commands
that my app is passing to the elasticsearch. It is because of our client
requirements regarding ability to track user requests through all system
components. After quick look into java api I do not see any possibility to
put additional field with requestId to the query builder. Can you confirm
that? What do you suggest to do then? Would it be easier to write
elasticsearch plugin adding this feature or should we just do some dirty
hacks like modifying query to contain request id but in the way that it
will not affect returned hits.

Best regards,
Krzysztof

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7d73dabc-a274-4488-bf03-709d41c37ccd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

I think you should solve that on application level.
Though could this help? Elasticsearch Platform — Find real-time answers at scale | Elastic

Still, you have to record that somewhere.

Not sure I helped... :frowning:

David

Le 19 nov. 2014 à 11:58, krzysztof.dawid.makowski@gmail.com a écrit :

Dear community,

I am looking for a feature which allows to attach requestId to the commands that my app is passing to the elasticsearch. It is because of our client requirements regarding ability to track user requests through all system components. After quick look into java api I do not see any possibility to put additional field with requestId to the query builder. Can you confirm that? What do you suggest to do then? Would it be easier to write elasticsearch plugin adding this feature or should we just do some dirty hacks like modifying query to contain request id but in the way that it will not affect returned hits.

Best regards,
Krzysztof

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7d73dabc-a274-4488-bf03-709d41c37ccd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/D41A5CEF-9CB2-4DC8-AF3B-A87F813CC2E3%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

3

Hmm

Thanks for the answer but it unfortunately did not help me. What I would
like to achieve is ability to group logs from various system components.

Assuming that we have a system containing 3 components:

A - HTTP frontend
B - My App
C - Elasticsearch

and communication chain is as follows:

CLIENT -> A (generation of unique request id happens here) -> B (with given
request id) -> C (with given request id)

Now if there is an exception or warning logged in one of those systems it
will be in the context of the generated unique request id. So assuming that
we have centralized logging infrastructure it is very easy now to track the
root causes and side effects of problems that occur in the system.

On Wednesday, November 19, 2014 12:53:28 PM UTC+1, David Pilato wrote:

I think you should solve that on application level.
Though could this help?
Elasticsearch Platform — Find real-time answers at scale | Elastic

Still, you have to record that somewhere.

Not sure I helped... :frowning:

David

Le 19 nov. 2014 à 11:58, krzysztof.da...@gmail.com <javascript:> a écrit :

Dear community,

I am looking for a feature which allows to attach requestId to the
commands that my app is passing to the elasticsearch. It is because of our
client requirements regarding ability to track user requests through all
system components. After quick look into java api I do not see any
possibility to put additional field with requestId to the query builder.
Can you confirm that? What do you suggest to do then? Would it be easier to
write elasticsearch plugin adding this feature or should we just do some
dirty hacks like modifying query to contain request id but in the way that
it will not affect returned hits.

Best regards,
Krzysztof

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7d73dabc-a274-4488-bf03-709d41c37ccd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/7d73dabc-a274-4488-bf03-709d41c37ccd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/02474d21-19eb-4420-9971-0ad48c2cb864%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi Krysztof,

I'm not 100% sure if this will help, but we are adding tracing and error
capture to SPM http://sematext.com/spm/. I think that may do what you
are after. In the mean time you could try setting up something like Zipkin.

Otis

Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

On Wednesday, November 19, 2014 8:02:03 AM UTC-5, krzysztof.da...@gmail.com
wrote:

Hmm

Thanks for the answer but it unfortunately did not help me. What I would
like to achieve is ability to group logs from various system components.

Assuming that we have a system containing 3 components:

A - HTTP frontend
B - My App
C - Elasticsearch

and communication chain is as follows:

CLIENT -> A (generation of unique request id happens here) -> B (with
given request id) -> C (with given request id)

Now if there is an exception or warning logged in one of those systems it
will be in the context of the generated unique request id. So assuming that
we have centralized logging infrastructure it is very easy now to track the
root causes and side effects of problems that occur in the system.

On Wednesday, November 19, 2014 12:53:28 PM UTC+1, David Pilato wrote:

I think you should solve that on application level.
Though could this help?
Elasticsearch Platform — Find real-time answers at scale | Elastic

Still, you have to record that somewhere.

Not sure I helped... :frowning:

David

Le 19 nov. 2014 à 11:58, krzysztof.da...@gmail.com a écrit :

Dear community,

I am looking for a feature which allows to attach requestId to the
commands that my app is passing to the elasticsearch. It is because of our
client requirements regarding ability to track user requests through all
system components. After quick look into java api I do not see any
possibility to put additional field with requestId to the query builder.
Can you confirm that? What do you suggest to do then? Would it be easier to
write elasticsearch plugin adding this feature or should we just do some
dirty hacks like modifying query to contain request id but in the way that
it will not affect returned hits.

Best regards,
Krzysztof

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7d73dabc-a274-4488-bf03-709d41c37ccd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/7d73dabc-a274-4488-bf03-709d41c37ccd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8b04393d-900d-4283-aae2-c8e5c616ff92%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

One possibility is to use search templates

The template names could be used as request ids.

But, if I understand your scenario, the real question is about how to
identify clients (e.g. by the node and socket connection they use, or more
precise, the netty channel ID), and compose a client ID with the request
IDs the client submits throughout the request life cycle. Currently, there
is no internal mechanism how Elasticsearch could associate clients with
their requests ("all search requests are equal").

Jörg

On Wed, Nov 19, 2014 at 11:58 AM, krzysztof.dawid.makowski@gmail.com
wrote:

Dear community,

I am looking for a feature which allows to attach requestId to the
commands that my app is passing to the elasticsearch. It is because of our
client requirements regarding ability to track user requests through all
system components. After quick look into java api I do not see any
possibility to put additional field with requestId to the query builder.
Can you confirm that? What do you suggest to do then? Would it be easier to
write elasticsearch plugin adding this feature or should we just do some
dirty hacks like modifying query to contain request id but in the way that
it will not affect returned hits.

Best regards,
Krzysztof

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7d73dabc-a274-4488-bf03-709d41c37ccd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/7d73dabc-a274-4488-bf03-709d41c37ccd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHr7uffbcc2YHXCGKn-xpai_%3DPKuVxGx%3D9iVW3CmJz7Jw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.