Routing with Nginx

Elastic Observability APM
1

Thanks for the product. I'm in the process of attempting to create a POC that will handle custom routing for the APM Agents.

A couple of questions.

I am trying to forward the secret token from the APM Agent => APM Server => Nginx. Is this possible? I would like to decode the token within nginx and then determine the appropriate route based off of the tokens contents.

If this is not possible, is it possible to dynamically update the Dictionary of Path Parameters dynamically based on the input from agent? For instance, setting the the serverURL = "http://apm:8200?routing=foo"

Thanks

2

Hi Steve,

Can you please elaborate a bit on what you're trying to do? It's not clear to me what your goal is, so I'm not sure how to answer the rest of your question.

If this is not possible, is it possible to dynamically update the Dictionary of Path Parameters dynamically based on the input from agent? For instance, setting the the serverURL = "http://apm:8200?routing=foo"

This will really depend on the agent, as that behaviour is not documented. The Go agent will include any query parameters you specify in ELASTIC_APM_SERVER_URL, but I don't know about the other agents.

3

Andrew,

I'm attempting to implement Multi-tenancy with APM Server / UI. Following this blog the recommendation is to use custom routing. https://www.elastic.co/blog/found-multi-tenancy.

My requirement is one APM server that can route multiple agents to isolated data within Elastic without creating an index per tenant (again per the blog). And then only show the data associated with an agents id (in this case a tenant id). We can not have tenant A seeing tenant B's data.

This works as expected if I place a static value in apm-server.yml (like below), but unfortunately, I do not see a way for this "value" to be dynamic based off a token, or an incoming query param. This is why I'm looking at alternatives.

  # Dictionary of HTTP parameters to pass within the url with index operations.
  # parameters:
  #   routing: "foobar"

The thought process is to have the APM Agent pass the secretToken through to nginx, decode the token and pull an id out to _route the request to elastic .. http://elk:9200?routing=foo.

Additionally, I'm using the node apm agent, and it doesn't appear to pass these queries off to the APM server, actually the APM server does not recognize the request as far as I can tell, when appending query params by watching the logs.

If you know of any "best practices" for multi-tenancy with APM that would be great to hear, or any workarounds to what I'm attempting to accomplish.

Thanks

4

I'm attempting to implement Multi-tenancy with APM Server / UI. Following this blog the recommendation is to use custom routing. https://www.elastic.co/blog/found-multi-tenancy .

My requirement is one APM server that can route multiple agents to isolated data within Elastic without creating an index per tenant (again per the blog). And then only show the data associated with an agents id (in this case a tenant id). We can not have tenant A seeing tenant B's data.

I see, thanks. To answer your original question: the secret token is strictly between the agent and the server, and is not available for enriching the events.

If you trust the tenants, then I think you could do something like: tag the events with a special tag like "tenant_id", and then use the rename processor to rename the field to "_routing". I haven't tried this myself.

5

Thanks Andrew, I'll give it a try!

6

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.