# Retrieve geolocation sampled data

**URL:** <https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432>\
**Category:** Elasticsearch\
**Created:** [July 19, 2019, 5:15pm UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432 "2019-07-19T17:15:12Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![MarcoAbi](https://avatars.discourse-cdn.com/v4/letter/m/dfb087/32.png) [@MarcoAbi](https://discuss.elastic.co/u/MarcoAbi)\
**Post date:** [July 19, 2019, 5:15pm UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432/1 "2019-07-19T17:15:12Z")

</div>

Hello,  
I have an index that collects 1 geospatial point data every second per each device (eg, a moving car).  
Now, I need to collect the geopoint in order to plot the path that the given vehicle has made. However, plotting a path of 2 hours long with a such frequency ends up with 3600\*2 datapoints, that is definitely too much for this use case (however, I do need such resolution for other use cases).  
Is there a way to query the index and retrieve only a sample of those 7200 points?

Thanks a lot!  
Marco

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [July 19, 2019, 5:37pm UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432/2 "2019-07-19T17:37:03Z")

</div>

> [@MarcoAbi](#):
>
> Is there a way to query the index and retrieve only a sample of those 7200 points?

We have a `sampler` aggregation which can take the top N hits and feed them to a contained child aggregation.  
Additionally there's a [diversified\_sampler](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-diversified-sampler-aggregation.html) which may be useful to ensure the selection of docs is not focused in any one particular time range or location.

---

<div class="post-metadata">

**Author:** ![MarcoAbi](https://avatars.discourse-cdn.com/v4/letter/m/dfb087/32.png) [@MarcoAbi](https://discuss.elastic.co/u/MarcoAbi)\
**Post date:** [July 19, 2019, 9:02pm UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432/3 "2019-07-19T21:02:25Z")

</div>

Wonderful!

Thanks a lot Mark.

Elastic is a masterpiece!

---

<div class="post-metadata">

**Author:** ![MarcoAbi](https://avatars.discourse-cdn.com/v4/letter/m/dfb087/32.png) [@MarcoAbi](https://discuss.elastic.co/u/MarcoAbi)\
**Post date:** [July 22, 2019, 2:23pm UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432/4 "2019-07-22T14:23:07Z")

</div>

Hello Mark,

I'm experiencing problems with the diversified\_sampler query.  
This is the query I'm setting up:

```
GET gps_data/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term" : {
            "deviceId": 123
          }
        },
        {
          "range": {
            "start": {
              "gte": 1562047171000,
              "lte": 1562047514000,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "filter": [],
      "should": [],
      "must_not": []
    }
  },
  "aggs": {
        "my_unbiased_sample": {
            "diversified_sampler": {
                "shard_size": 200,
                "field" : "eventTime"
            },
            "aggs": {
                "keywords": {
                    "significant_terms": {
                        "field": "location"
                    }
                }
            }
        }
    }
}

```

Elastic replies back that "significant\_terms aggregation cannot be applied to field [location]. It can only be applied to numeric or string fields." I'm wondering if I'm setting up the query correctly or actually there is this limitation in the query.

Thanks!  
Marco

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [July 22, 2019, 3:12pm UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432/5 "2019-07-22T15:12:36Z")

</div>

> [@MarcoAbi](#):
>
> Elastic replies back that "significant\_terms aggregation cannot be applied to field [location].

Wasn't the goal to put your geo aggregation under the sampler?  
You may need to have a higher-granularity field for diversification too - if the accuracy is millisecond level you'll only be limiting the number of docs considered per millisecond. You might need to use a [script](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-diversified-sampler-aggregation.html#_scripted_example) to "round up" the times to hours or minutes or whatever suits as your unit for de-duplicating

---

<div class="post-metadata">

**Author:** ![MarcoAbi](https://avatars.discourse-cdn.com/v4/letter/m/dfb087/32.png) [@MarcoAbi](https://discuss.elastic.co/u/MarcoAbi)\
**Post date:** [July 23, 2019, 6:48am UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432/6 "2019-07-23T06:48:48Z")

</div>

Ok, got your point.

I have to refactor the index a bit than, I want to avoid scripted fields to maintain performances high enough.

Meanwhile I’m using the functional query with a random function. This is creating the uniform scoring that allows me to drop out enough data and have a rough good quality samples.

Thanks again for your support.

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [July 23, 2019, 11:20am UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432/7 "2019-07-23T11:20:27Z")

</div>

> [@MarcoAbi](#):
>
> I’m using the functional query with a random function.

Sounds like a good approach.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 20, 2019, 11:20am UTC](https://discuss.elastic.co/t/retrieve-geolocation-sampled-data/191432/8 "2019-08-20T11:20:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
