Logs ML event.dataset type error

Elastic Observability Logs
1

I am trying to setup Logs ML anomaly detection and getting the following error

at least one index has a field event.dataset without the correct type

image

Here is the typings for my index, I have seen various questions about this on here, and people have said ensure its got the right type, but never clarified which type it needs to be

{
  "logs-kiosk" : {
    "mappings" : {
      "properties" : {
        "@timestamp" : {
          "type" : "alias",
          "path" : "timestamp"
        },
        "agent" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "agentId" : {
          "type" : "integer"
        },
        "event" : {
          "properties" : {
            "dataset" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            }
          }
        },
        "level" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "message" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "payload" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "source" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "timestamp" : {
          "type" : "date"
        }
      }
    }
  }
}
2

Most of the { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } should be { "type": "keyword", "ingore_above": 256 } for this to work properly, there are some exceptions (see ECS reference below). The problem being that when event.dataset is a text field it can't be used with aggregations.

Logs (and the ML jobs) expects the schema to follow ECS: ECS Field Reference | Elastic Common Schema (ECS) Reference [1.12] | Elastic

3

Thanks, I will give this a try, pretty sure it will work, but will report back. Also, thanks for the schema.

Update - This worked, Thanks again.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.