Any Reason traces-apm@template is using Standard Index mode instead of timeseries or logsdb

we are capturing enterprise java traces, and the trace is filling up all the spaces on my cluster, its generating 1.4TB daily and i cant keep up, then upon checking there is no compression on the created index and its just a standard index, is there any reason this was used as default and if i can change the template to timeseries or logsdb ?

Kibana version: 9.2

Elasticsearch version: 9.2

APM Server version: elastic-agent 9.2

APM Agent language and version: java 1.54

@Serak_Shiferaw

Great question / common question

Legacy agents are standard mode.

In short work was stopped on making sure that logs DB would be fully compatible/ tested and supported for legacy APM agents because focus is on OTEL.

EDIT / ADDED: therefore traces with the legacy agents are only in standard mode.

My understanding is there's a few places in the curated UI that will not work properly if you convert your traces to logs DB... You can certainly try it but it would not be supported if you had issues.

Out of the box OTEL traces are logsdb.

on my server it says otherwise, its not logsdb its just a standard index, not sure what i did wrong

@Serak_Shiferaw

Sorry, maybe I was not clear ... Oh I see. I was not LOL I left out the word NOT :slight_smile:

What you have is exactly correct.

Traces from elastic legacy agents are not logsDB that's exactly what you're showing right there and what I said about apologies if it was confusing.

traces-apm-default are traces from the legacy agents.

When you switch to OTEL agents then they will be logs DB automatically.

Apologies again for the confusion.

ah you mean this Open Telemetry agent from here : Set up the EDOT Java Agent | Elastic Distribution of OpenTelemetry Java release notes ? ok i will try that but not sure if IBM JVM is supported, i am on IBM AIX, with Websphere 9 i.e IBM JVM

Not just EDOT and Upstream OTEL Agent should work as well.. but you will need to pass it through an OTEL Collector to then send to Elastic.

You should be able to use this flow

EDOT / OTEL SDK (otlp) -> EDOT Collector / OTEL Col -> ECH (Now, OTEL Native Schema and logsdb)

:unamused_face: how does this make OTEL accessible and open and better ? i mean i can no longer use Fleet Servers ?

Hi @Serak_Shiferaw

OTEL is an industry standards initiative that Elastic is fully embracing and is on a Journey. Elastic legacy agents / component will be around for a while, but over time as OTEL matures they will become fully OTEL.

That said, Central Management of Agents using the new OTEL standard OpAMP s on the Roadmap, I can not comment when that will be delivered.

Curious which feet features you are using with APM traces?

1 Like

currently java agent is sending the traces to fleet servers which is running apm service, i want to utilize same workflow where otel is metrics is sent to fleet servers running specific service related like otel collector, fleet is just easy to manage and configure

Right, so you are using Fleet to manage the APM Integration Server

At some point, there will bea Centrally Managed OTEL Collector that will serve the same purpose.

You can use these ingestion paths for now if you like

B) OTEL SDK (otlp) -> APM Server -> ECH (Now, but Elastic ECS APM Format) :white_check_mark:
C) EDOT SDK (otlp) -> APM Server -> ECH (Now, but Elastic ECS APM Format) :white_check_mark:

When I say Elastic ECS APM Format that means the data ends up in ECS Elastic Common Schema not OTEL Native Semantic Convention