Character limit in fields longer then 1024 (truncated with ellipsis)

Elastic Observability APM
1

Hi/

i'm having an issue with certain logs fields being longer than 1024 chars.
When searching for the relevant log it comes up, but the relevant field is truncated, and the remainder of the text is replaced with an ellipsis character.

(as in: field/keyword: "loglog .. .. log…"
where the '…' char appears as the 1025 char in my log field).

We're using an APM Agent in our .Net application (classic / aspx),
and are querying the data using kibana's "discover" area.

Note- we're not getting errors when posting logs, but rather that they come up "partial" when we're querying them.

From reading up on this in various places:

This would seem to be configurable somewhere somehow. Unfortunately i couldn't figure out where this change should be applied --- Tried looking at our .Net app's web.config file, the .cs file where we implement our Application_Start() and the relevant Agent APM instance....

Couldn't find a relevant configuration option in any of them, like a truncateStringsAt or sanitization options, nor a maxLength like property for the server's json config...

Where is this limitation coming from?
Is it a server-side config? (where the logs are sent to?)
Is there some other config on the App / Object's side that i'm missing or unfamiliar with?

Any help / pointers would be greatly appreciated !

2

Hello,

Can you please check the mapping of the field which is getting truncated? Please take a look at this post and see if your issue is similar?

Thanks
Bhavya

3

Hi @bhavyarm , thank you for the reply!

Will check the "truncate:maxHeight" setting on the server's advanced settings.
However, I think the situation is different in 2 ways from the post provided as a reference.

1st, the interface used by the user in that post is the "Dashboard", where as we're using the "Discover" mechanism to run our queries.

2nd, that user's issue seems to revolve around FINDING the info in question, due to a configurable limit, whilst our issue is with READING said data, which as mention seems to be "cut off" at the 1024th char.

so i'm unsure if this will provide a solution, but will update back promptly once we change it.

in regard to your request we "check the mapping of the field", could you point me towards any resource explaining how to access these mappings? (are they maintained on the sending / receiving side, how to i check them directly etc...)

and thanks again ! :smiley:

4

Hello,

Here you go to check the mapping - Get mapping API | Elasticsearch Guide [7.14] | Elastic

I also got feedback from an engineer about having to ask the apm to team to take a look at this. I am transferring your post to them.

Thanks
Bhavya

5

Hi @GiladanoN,

The APM agents send data to APM server in formats that conform to JSON schemata. For example, the schema for spans is

The schema defines a max length of 1024 for many of the string fields, in order to keep APM index sizes from growing large.

The .NET agent (and I think other agents too) hard limit the size of properties on trace types like Span and Transaction in line with the schema, and add an ellipsis to indicate a value has been truncated. Generally, fields with a max length are not configurable.

Are there particular fields you're using where max length configuration would be useful?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.