How remove APM application from Kibana UI

Elastic Observability APM
1

I have two spaces in Kibana (Prod and Stage) but I think APM is global in all spaces right? I can see my java applications that are staging in the production space and viceversa, that's correct?

Or can I have staging applications only in "Stage Space" and productions applications in "Prod Space"? the index form apm is the same.

However I see one wrong name of my application in APM section and I would like delete it, but I tried with this request but didn't work

POST /apm-*/_delete_by_query
{
  "query": {
"bool": {
  "must": [
    {
      "term": {
        "service.name": {
          "value": "myapp1"
        }
      }
    }
  ]
}
  }
}

The response:

{
  "took": 197,
  "timed_out": false,
  "total": 0,
  "deleted": 0,
  "batches": 0,
  "version_conflicts": 0,
  "noops": 0,
  "retries": {
    "bulk": 0,
    "search": 0
  },
  "throttled_millis": 0,
  "requests_per_second": -1,
  "throttled_until_millis": 0,
  "failures": [
    {
      "index": "apm-7.3.2-span-000007",
      "shard": 0,
      "node": "N0Piisd9QUGe_f31WC43Ww",
      "reason": {
        "type": "circuit_breaking_exception",
        "reason": "[parent] Data too large, data for [<transport_request>] would be [412809874/393.6mb], which is larger than the limit of [394910105/376.6mb], real usage: [412807528/393.6mb], new bytes reserved: [2346/2.2kb], usages [request=0/0b, fielddata=52530/51.2kb, in_flight_requests=303568/296.4kb, accounting=15940953/15.2mb]",
        "bytes_wanted": 412809874,
        "bytes_limit": 394910105,
        "durability": "PERMANENT"
      }
    }
  ]
}

I don't know what I can do to delete one old specific service.

Thank you so much

2

Hi :wave:

Your query look correct. The problem is that you are hitting a memory limit. You could try increasing the limit which is currently 376.6mb.

You can also call the following API to inspect circuit breakers:

GET /_nodes/stats/breaker
3

Hi @sqren!

What node do you mean? elasticsearch o apm server?

Thanks

4

Elasticsearch seems to be running out of memory when you run your query.