Apm for django, filter errors by cusotm field

Elastic Observability Logs
1

I have set up elastic apm for a django project with logging (elasticapm.contrib.django.handlers.LoggingHandler). Upon logging a message as follows:

logger.exception(
    custom_logging_message,
    exc_info=True,
    extra={
        'status_code': status_code
    },
)

it correctly records the error document, including the extra field status_code under error.custom.status_code, so far so great.

but i cannot use it in a filter (or DSL query), like error.custom.status_code >= 500

is this because the field is not indexed? i tried to find all indexed fields via GET _my_error_index/_mapping?pretty and indeed it is not among the "mappings". how can i use my custom logging data in filters or DSL queries? thank you

2

Hi @k_cf Can you share the output you got from the mapping command? Even though likely status_code is index as a keyword instead of an integer, I would still expect it to show up. Ideally could you also share a sample document that is indexed?

Two general recommendations:

  • For logs, ship your data to the data stream naming scheme. In your example, this might be logs-my_error-default. This gives you all ECS mappings by default
  • Taking about ECS, use ECS fields if possible so you get the correct mapping automatically. In your scenario, it seems to be custom status code. If it is shipped as long, with this template you should still get the right mapping automatically.