Elastic APM java : Get Active serviceName

[Namita_Jaokar]

Hi,

I have a scenario wherein I need to get the list of active services in elasticsearch (ServiceName that would be bound to the Elastic APM Jar in case of java , .dll in case of .net, etc ) . I'll need these service Names to be showcased my application.

For the same I came across this link : Swagger UI

The above has some api's that gives the status of the agent, But probably these are Elastic Agents and not Elastic APM agents/Services as per my understanding. Please confirm whether my assumption on the same is correct.

I have Also tried installing heartbeat in my local, But again in this case will I be able to fetch the Elastic APM Agents service name?
Currently in the installation process I am getting below error on running the test command (heartbeat test config -e) . Although , the test runs successful.

{"log.level":"warn","@timestamp":"2022-12-15T17:05:55.130+0530","log.origin":{"file.name":"beater/heartbeat.go","file.line":69},"message":"could not connect to ES for state management during initial load: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://localhost:9200: Get "https://localhost:9200": x509: certificate signed by unknown authority]","service.name":"heartbeat","ecs.version":"1.6.0"}

My heartbeat.yml has below output.elasticsearch properties:
output.elasticsearch:
hosts: ["localhost:9200"]
protocol: "https"
username: "elastic"
password: "mypassword"

Can someone suggest an approach for the same, Are there any API's or Elastic Query to fetch the active elastic APM services

Thanks & Regards,
Namita

[Sylvain_Juge]

Hi @Namita_Jaokar ,

First make sure that you can properly connect to ES cluster, executing a GET https://localhost:8200/ should not return an exception. In your case it seems you have a certificate issue, probably because Elasticsearch is using a self-signed certificate and your application rejects it.

Then, once you can connect, you should be able to query the APM documents and filter the most recent values for service.name.

What confuses me here is that you mention heartbeat configuration, whereas trying to retrieve the services names captured by APM seems to be doable with a query directly into Elasticsearch.

[Namita_Jaokar]

Hi Sylvain,

Thank you for the quick response. I was trying out different ways to get the service.name for Elastic APM agents and hence thought of giving heartbeat a try.

Coming further to your response, With EQL I should be able to get all the active service.name configured with the APM Agent? Or it will give me all the service.name irrespective of it being in active or inactive state.

Best Regards,
Namita

[Sylvain_Juge]

Hi,

I don't know what it would look like with EQL, but you can simply query Elasticsearch API, for example with this in Dev Tools > Console you should get the list of the services that sent some data within the last 7 days:

GET traces-apm*,apm-*,metrics-apm*,apm-*,logs-apm*,apm-*/_search
{
  "size": 0,
  "fields": [
    "service.name",
    "@timestamp"
  ], 
  "query": {
    "bool": {
      "filter": [
        { "range": {
          "@timestamp": {
           "gte": "now-7d/d"
          }
        }}
      ]
    }
  }, 
  "_source": false,
  "aggs": {
    "service_name": {
      "terms": {
        "field": "service.name",
        "size": 10
      }
    }
  }
}

Sample output:

{
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 97,
    "successful": 97,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "service_name": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "opbeans",
          "doc_count": 9240
        },
        {
          "key": "opbeans-stream-logs",
          "doc_count": 7770
        },
        {
          "key": "unknown-java-service",
          "doc_count": 156
        }
      ]
    }
  }
}

[Namita_Jaokar]

Thank You @Sylvain_Juge .
I had tried similar query by providing timestamp range and was able to get the required service name.

Thanks for providing the suggestion.

Best Regards,
Namita

[Namita_Jaokar]

Hi @Sylvain_Juge ,

I tried implementing the query using elasticsearch java client. In the initial research , I found few references on internet that use TermsBuilder for the aggregation part of the query ( "aggs": { "service_name": { "terms": { "field": "service.name", "size": 10 } } } )

However, When I try the same there are no imports found for TermsBuilder.
Apart from this, I also tried using AggregationBuilder but that gives me compilation error as there's no inbuilt method in the builder AggregationBuilder.terms("service_name").

I am unable to proceed with the query as I am not able to find a suitable way to add aggregations to my query in java. Please suggest.

I am using below two maven dependencies:

• elasticsearch-rest-high-level-client version 7.17.8
• elasticsearch-java version 8.5.3

While declaring the restclient I am getting below issue

Am I using the correct dependencies for java client and highlevel rest client. Please advise.

Thanks & Regards,
Namita

[Sylvain_Juge]

Hi @Namita_Jaokar ,

Unfortunately, I am really not familiar with the Elasticsearch Java client nor its API.
What I know though is that you should use the version that matches your stack version and do not mix 7.x and 8.x like you did.

Also, I would recommend trying to stick to the 8.x version to make your code more future-proof. While the 8.x Java client relies on the "REST high level" client, it provides a more modern API. see docs for more information here.

[Namita_Jaokar]

Thanks @Sylvain_Juge . Will go through the documentation link.

Thanks & Regards,
Namita

[Namita_Jaokar]

Hi @Sylvain_Juge ,

GET traces-apm*,apm-,metrics-apm,apm-,logs-apm,apm-*/_search
{
"size": 0,
"fields": [
"service.name",
"@timestamp"
],
"query": {
"bool": {
"filter": [
{ "range": {
"@timestamp": {
"gte": "now-7d/d"
}
}}
]
}
},
"_source": false,
"aggs": {
"service_name": {
"terms": {
"field": "service.name",
"size": 10
}
}
}
}

The above query provides me the list of active serivce.name. If in case I want to provide the list of service.name to the EQL how can I implement that?

I tried using below query.

GET /metrics-apm*/_search
{
"query" :
{
"terms"{
"service.name" : [
"service_1",
"service_2",
"service_3"
]
}
}
}
But in this case I even want to add the time range which does not go with the above query I tried. Is it possible to fetch Multiple values with range in EQL? Can it be done using aggregations ?

Best Regards,
Namita