APM recognizes all logs as errors

[fr0der1c]

I'm using APM on a flask application with logbook.

Here is how I register LogBook handlers:

from elasticapm.handlers.logbook import LogbookHandler as ElasticHandler

...

    if not offline:
        # Sentry
        sentry.init_app(app=app)
        sentry_handler = SentryHandler(sentry.client, level='WARNING') 
        logger.handlers.append(sentry_handler)

        # Elastic APM
        apm = ElasticAPM(app)
        elastic_handler = ElasticHandler(client=apm.client, bubble=True)
        logger.handlers.append(elastic_handler)

Here is how I use logger:

logger.info('App created with `{0}` config'.format(app.config['CONFIG_NAME']), stack=False)

Since I already have Sentry, I just want to use Elastic APM to centralize my logs. But on the APM panel in Kibana, all the logs as treated as errors. Is there any way to adjust this behavior? Thanks!

[beniwohli (Benjamin Wohlwend @ APM)]

Hi @fr0der1c

for centralized logging, we recommend to use a filebeat, possibly in combination with Logstash and/or Elasticsearch Ingest Node. This is a much more efficient way to get log files into Elasticsearch than using the Elastic APM Python agent, as those tools are specifically optimized for log shipping.

The log handlers that come with the APM Python agent are better suited for logging errors, as they collect a large amount of contextual information (like stack traces, local variables, etc). These aren't needed in most cases for informational logs and therefore only create overhead both in your app as well as when indexing and storing data in Elasticsearch.