Elastic APM Index management mystery

[BartoszDWH (Bartosz Gajda)]

Hello,

We're using APM as our application telemetry sink. I've read articles about index management and found out that everything works as it should except for one thing which bothers me and creates disk space outage.

APM feeds apm-default indices via traces-apm-default data stream and I managed to find all ILMs for them, but there are also several indices that are fed by APM but I cannot find anything related to that (no data stream, no template, and especially no ILMs). The pattern of mystery is serviceName-environment-year-month.

Example: myApp-prod-2024-03

Currently, I am deleting the ones which are >3 months old. I want to tune/create a policy for these indices so we won't run out of disk memory in the future.

[stephenb (Stephen Brown)]

Hi @BartoszDWH Welcome to the community

I am not clear if that is an exact or "kinda-like" example

Os that the full data stream name? or partial

@BartoszDWH

myApp-prod-2024-03

If that is a full datastream name, APM does not create that.

However, APM can create service-specific metrics data streams if you are sending custom metrics.

This is an example of a data stream created by apm for custom metrics

metrics-apm.app.cardatabase_back_end-default

[Carson_Ip]

Hi @BartoszDWH ,

Do you mind sharing the version you are on? I do not expect indices like myApp-prod-2024-03 created by APM, at least not on 8.x. Also, please share the complete data stream name, just to be sure.

Thanks.

[BartoszDWH (Bartosz Gajda)]

Thanks for your replies!

You're right. I see those indices, but what I am trying to resolve is these ones:

while myApp is the service name.

I think I got a clue, isn't just raw logs written to APM stored here? Using discovery I can found that are just raw logs produced by app.

App is dotnet application with Serilog along with APM Sink. APM is in newest version v8.12.2. I don't see any possible configuration down here Serilog | APM .NET Agent Reference [1.x] | Elastic

[stephenb (Stephen Brown)]

Those are not APM created indices...

myApp-production-2024-03

Did you simply look at a few documents in the index, often how/what is being used to ingest them they are being sent will be part of the document data .