Message field not searchable in container logs from Kubernetes integration

Elastic Stack Kibana
,
1

We are using Elatic Cloud to collect logs from a Kubernetes cluster using fleet-managed Elastic Agents.

When enabling the Kubernetes integration in a policy, the Agents automatically pick up container logs. That is really useful as that means all the logs from our apps flow into Elastic.

However, when searching I noticed I do not get any matches on individual words in the message field. Looking at the index template: logs-kubernetes.container_logs, it has no mapping for the message field. The result is that the message field is mapped as "keyword" instead of "match_only_text".

Looking at the template from another integration (logs-system.system), that one includes the following in it's template:

    "message": {
      "type": "match_only_text"
    }

Question: is the message field omitted from the Kubernetes integration intentionally?

If not, would it make sense to add that?

Or would it be an idea to add the 'data-streams-mappings' managaged component templates as a component in the managed logs-kubernetes.container_logs template? That component template includes a nice dynamic template that matches the message field and makes it match_only_text.

{
  "dynamic_templates": [
    ...
    {
      "match_message": {
        "mapping": {
          "type": "match_only_text"
        },
        "match_mapping_type": "string",
        "match": "message"
      }
    },
    ...
2

I did some more digging and found the source for the kubernetes Fleet integration. The datastream definition does seem to mention the message field. Specifically with an external: ecs value.

The spec documentation seems to suggest that the field definition will be taken from the ecs field definitions. Although I haven't found the bit that implements that logic yet.

The definition of the system.system data_stream seems to be identical, so I don't understand why that one is using match_only_text and the template for kubernetes.container does not have a definition for message and therefor falls back to the default keyword.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.