Basic runtime fields to pull data from message field

[randomnamegenerator]

Hello all,

I am trying to create a series of runtime fields to extract additional data fields from the message field. If it makes any difference this is a EFK stack running in K8s.

For example I want to create level field for our incoming logs (info,warn,error etc) and then aggregate each level

This is our log format

2024-03-13 07:36:59.195 [123] INFO  blablabla: (unauthorized) 200 /api/blablatook 00:00:00.0001609

If anyone could recommend the correct expression that would be greatly appreciated.

I have read through the documentation but when I tried to reference the message.keyword field it appears as though its not found?

Thanks in advance

[jughosta (Julia Rechkunova)]

Hi @randomnamegenerator,

The expected format to access a field value inside a runtime field script is doc["message.keyword"].value Manage data views | Kibana Guide [master] | Elastic

There are also other ways to parse logs. I would recommend to check these docs:

• Parsing Logs with Logstash | Logstash Reference [8.13] | Elastic
• Filebeat quick start: installation and configuration | Filebeat Reference [8.13] | Elastic
• Parse and organize logs | Elastic Observability [8.13] | Elastic
• Example: Parse logs in the Common Log Format | Elasticsearch Guide [8.13] | Elastic