# How to correctly use lat\_long detector on geo.location from azure.signin dataset

**URL:** <https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388>\
**Category:** Kibana\
**Tags:** elastic-stack-machine-learning\
**Created:** [May 19, 2020, 6:08pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388 "2020-05-19T18:08:43Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [May 19, 2020, 6:08pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/1 "2020-05-19T18:08:44Z")

</div>

Hello,

When I try to create a machine learning job with lat\_long detector, for some reason, the job is not starting to process records.

Query:

`{"bool":{"must":[{"match_all":{}}],"filter":[{"match_phrase":{"event.dataset":"azure.signinlogs"}}],"must_not":[]}}`

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/e/9e4c8404d643379315bc6471974ef630c1918f19.png)  
Anyone an idea why the ml job does not seem to start processing records? There is definitely signin data in our filebeat index.  
Any sugggestion is welcome to make this work. I'm trying to replicate some Azure Sentinel functionalities and find out when a user is signing in from a anomalous location.

(first time I'm using the lat\_long detector so it's probably a rooky mistake)

The datafeed preview is also empty:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/4/a4537635081505e569892a424c075fc3740c5ead.png)

Grtz

Willem

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [May 19, 2020, 6:37pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/2 "2020-05-19T18:37:07Z")

</div>

From: [Geographic functions | Machine Learning in the Elastic Stack [7.7] | Elastic](https://www.elastic.co/guide/en/machine-learning/7.7/ml-geo-functions.html)

> The `field_name` that you supply must be a single string that contains two comma-separated numbers of the form `latitude,longitude` , a `geo_point` field, a `geo_shape` field that contains point values, or a `geo_centroid` aggregation. The `latitude` and `longitude` must be in the range -180 to 180 and represent a point on the surface of the Earth.

Can you verify the type of field your `geo.location` field is mapped as?

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [May 19, 2020, 7:01pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/3 "2020-05-19T19:01:38Z")

</div>

@richcollier Tx for the fast answer.

Fyi this on Elastic 7.6.1

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/3/b373a6f25fb4b8d533e4f5b48ff450637fa69da2.png)

```
"geo" : {
  "properties" : {
    "city_name" : {
      "type" : "keyword",
      "ignore_above" : 1024
    },
    "continent_name" : {
      "type" : "keyword",
      "ignore_above" : 1024
    },
    "country_iso_code" : {
      "type" : "keyword",
      "ignore_above" : 1024
    },
    "country_name" : {
      "type" : "keyword",
      "ignore_above" : 1024
    },
    "location" : {
      "type" : "geo_point"
    },
    "name" : {
      "type" : "keyword",
      "ignore_above" : 1024
    },
    "region_iso_code" : {
      "type" : "keyword",
      "ignore_above" : 1024
    },
    "region_name" : {
      "type" : "keyword",
      "ignore_above" : 1024
    }
  }
} 

```

This is coming from the azure.signin Filebeat event.dataset

Grtz

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [May 19, 2020, 8:15pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/4 "2020-05-19T20:15:14Z")

</div>

Ok - glad to see that it is truly a `geo_point`.

If you're seeing nothing in the datafeed preview you should probably test to see if your query of the raw data is working as expected. Take what you have as the datafeed query and try it as a standard `_search` (I'm guessing below that your index pattern is indeed `filebeat-*`, but modify if necessary):

```auto
GET filebeat-*/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        }
      ],
      "filter": [
        {
          "match_phrase": {
            "event.dataset": "azure.signinlogs"
          }
        }
      ],
      "must_not": []
    }
  }
}

```

Does that properly return the data you expect?

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [May 19, 2020, 9:04pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/5 "2020-05-19T21:04:51Z")

</div>

> [@richcollier](#):
>
> GET filebeat-\*/\_search { "query": { "bool": { "must": [{ "match\_all": {} }], "filter": [{ "match\_phrase": { "event.dataset": "azure.signinlogs" } }], "must\_not": } } }

@richcollier Yes the query returns the docs I expected (can't show them for privacy concerns etc) incl the `geo.location` field.

```
GET filebeat/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        }
      ],
      "filter": [
        {
          "match_phrase": {
            "event.dataset": "azure.signinlogs"
          }
        }
      ],
      "must_not": []
    }
  }
} 

```

I do use an alias filebeat though, but I've been doing that before, so I don't think that's the issue.

Grtz

Willem

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [May 19, 2020, 9:25pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/6 "2020-05-19T21:25:03Z")

</div>

Hmm...well there is nothing obvious that is wrong. I attempted a similar configuration using the `kibana_sample_data_ecommerce` data that ships with Kibana.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/c/ccd34e0163400e4c60e5ba9c8fd9830850fd0f68.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3ce7e581b66236cf7824b3c2068514461b1cdecf.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/9/49de24d338cc3fe1964a6cebba1c5d9aefa1bd9f.png)

Perhaps try it on the sample data and then compare with your "real" setup and see what is different? If you cannot figure it out after that, It will be hard to debug further in this setting. A proper Support Case would likely be necessary.

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [May 19, 2020, 9:28pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/7 "2020-05-19T21:28:38Z")

</div>

Getting some sleep now, I will try your sample Data suggestion and might make a case. Tx!

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [May 21, 2020, 7:30am UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/8 "2020-05-21T07:30:22Z")

</div>

@richcollier Started a "lat\_long(OriginLocation) by Carrier" on the flights sample dataset and it immediately started processing.

```
  "analysis_config": {
    "bucket_span": "15m",
    "detectors": [
      {
        "detector_description": "lat_long(OriginLocation) by Carrier",
        "function": "lat_long",
        "field_name": "OriginLocation",
        "by_field_name": "Carrier",
        "detector_index": 0
      }
    ],
    "influencers": [
      "OriginWeather",
      "Carrier",
      "DestWeather"
    ]
  }, 

```

Not sure what I'm doing wrong on my Azure signin logs.. Tried recreating the `lat_long("geo.location") by "azure.signinlogs.identity"` but again no records processed.

See 00547781

Grtz

Willem

---

<div class="post-metadata">

**Author:** ![richylyq](https://avatars.discourse-cdn.com/v4/letter/r/41988e/32.png) [@richylyq](https://discuss.elastic.co/u/richylyq)\
**Post date:** [May 21, 2020, 8:48am UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/9 "2020-05-21T08:48:11Z")

</div>

Hi,

Not sure if this helps but i was also doing the geo\_point stuff but it was with python. and i had to add the pipeline

```auto
# defining the ingest pipeline and the specific values 
p.put_pipeline(id='attachment', body={
    'description': 'Extract attachment information',
    'processors': [
        {
            "set": {
                "field": "location",
                "value": "{{Lat}}, {{Lon}}"
            }
        }
    ]
})

```

location here is a `geo_point`, and it takes in value from latitude and longitude

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [May 21, 2020, 4:34pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/10 "2020-05-21T16:34:59Z")

</div>

Forgot to ask this obvious question. Did you have your original job "look back" in time, or did you only kick it off in real-time?

---

<div class="post-metadata">

**Author:** ![willemdh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willemdh/32/16922_2.png) [@willemdh](https://discuss.elastic.co/u/willemdh)\
**Post date:** [May 22, 2020, 12:35pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/11 "2020-05-22T12:35:07Z")

</div>

Well I created the job 2 dyas or so after we started using the signing logs dataset. And configured it to start from the beginning of the data and continue. I attached a video to the case. Grtz

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 19, 2020, 12:39pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-lat-long-detector-on-geo-location-from-azure-signin-dataset/233388/12 "2020-06-19T12:39:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.