Why can't I use user.name as field in machine learning job, but the standard jobs can?

Elastic Stack Kibana
21

Here it is:

22

It really isn't obvious what's the issue here. Worst case scenario is that you always edit the JSON for the job on the last step of the wizard to change the field in the detector to be user.name. This will get around the UI restriction.

24

Somehow this does not work for me. I just tried editing (removing .keyword) this in the last step (summary page):

{
  "job_id": "testpws",
  "description": "",
  "groups": [],
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [
      {
        "function": "rare",
        "by_field_name": "user.name.keyword"
      }
    ],
    "influencers": [
      "user.name.keyword"
    ]
  },
  "data_description": {
    "time_field": "@timestamp"
  },
  "analysis_limits": {
    "model_memory_limit": "10MB"
  },
  "model_plot_config": {
    "enabled": false,
    "annotations_enabled": false
  }
}

It automatically removes the "by_field_name" field, like this:

{
  "job_id": "testpws",
  "description": "",
  "groups": [],
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [
      {
        "function": "rare"
      }
    ],
    "influencers": [
      "user.name"
    ]
  },
  "data_description": {
    "time_field": "@timestamp"
  },
  "analysis_limits": {
    "model_memory_limit": "10MB"
  },
  "model_plot_config": {
    "enabled": false,
    "annotations_enabled": false
  }
}
25

Well, not sure what's up with your setup and honestly, we're probably at the limit of what we can do here on a forum. A proper support ticket would normally be the next step but you're doing this as a project and not as a paid customer. In the meantime, your best bet is likely to bypass the UI altogether and just create your jobs with the API. So for example:

PUT _ml/anomaly_detectors/aaa
{
    "job_id": "aaa",
    "custom_settings": {},
    "description": "",
    "analysis_config": {
      "bucket_span": "15m",
      "detectors": [
        {
          "detector_description": "rare users",
          "function": "rare",
          "by_field_name": "user.name",
          "detector_index": 0
        }
      ],
      "influencers": [
        "user.name"
      ],
      "model_prune_window": "30d"
    },
    "analysis_limits": {
      "model_memory_limit": "11mb",
      "categorization_examples_limit": 4
    },
    "data_description": {
      "time_field": "timestamp",
      "time_format": "epoch_ms"
    },
    "model_plot_config": {
      "enabled": false,
      "annotations_enabled": false
    },
    "model_snapshot_retention_days": 10,
    "daily_model_snapshot_retention_after_days": 1,
    "results_index_name": "aaa",
    "allow_lazy_open": false
}

and

PUT _ml/datafeeds/datafeed-aaa
{
     "datafeed_id": "datafeed-aaa",
    "job_id": "aaa",
    "query": {
      "bool": {
        "must": [
          {
            "match_all": {}
          }
        ]
      }
    },
    "indices": [
      "winlogbeat-*"
    ],
    "scroll_size": 1000,
    "delayed_data_check_config": {
      "enabled": true
    }
}

Replacing all occurrences of aaa above with your desired job name.

Good luck!

27

This works for now. Thank you for the support.

28

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.