# Duplicate results when paging

**URL:** https://discuss.elastic.co/t/duplicate-results-when-paging/260895
**Category:** Elasticsearch
**Created:** [January 12, 2021, 8:01pm UTC](https://discuss.elastic.co/t/duplicate-results-when-paging/260895 "2021-01-12T20:01:40Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![noah-elastic](https://avatars.discourse-cdn.com/v4/letter/n/5e9695/32.png) [@noah-elastic](https://discuss.elastic.co/u/noah-elastic)
#### Post date: [January 12, 2021, 8:01pm UTC](https://discuss.elastic.co/t/duplicate-results-when-paging/260895/1 "2021-01-12T20:01:40Z")

</div>

Hello Folks --

I'm researching an interesting situation that one of our users pointed out. We have a somewhat simple query that searches on a couple fields. We are using size/from to allow the users to step through the pages. Some times the last page of results contains results that were included in a previous page. I turned on explain for the query and I see that in those cases the results are from different nodes:

```auto
    {
        "_shard": "[index-changed][1]",
        "_node": "PKznqHx5QfCRke-rsxzn-w",
        "_index": "index-changed",
        "_type": "_doc",
        "_id": "633637",
        "_score": 12.777646,

```

and

```auto
    {
        "_shard": "[index-changed][1]",
        "_node": "TzNHbXfHSvulWCJP6pXKow",
        "_index": "index-changed",
        "_type": "_doc",
        "_id": "633637",
        "_score": 12.64538

```

I understand that this is somewhat expected, and that I could use the preference query param to use the same shards. However it doesn't seem to make a difference, I still get results from both nodes, resulting in slightly different scores. What am I missing? How can I eliminate duplicates, and get consistent results when paging?

Thanks!

EDIT:  
Here's the query:

```auto
{
    "query": {
        "bool": {
            "must": {
                "multi_match": {
                    "query": "search text",
                    "fields": [
                        "field1",
                        "field1.english",
                        "field2^0.1"
                    ],
                    "type": "phrase",
                    "slop": 10
                }
            },
            "must_not": [
                {
                    "range": {
                        "ends_at": {
                            "lt": "2021-01-12T17:12:44.107Z"
                        }
                    }
                }
            ],
            "should": [
                {
                    "distance_feature": {
                        "field": "updated_at",
                        "pivot": "90d",
                        "origin": "now",
                        "boost": 0.5
                    }
                },
                {
                    "terms": {
                        "type": [
                            "TYPE2",
                            "TYPE2",
                            "TYPE3"
                        ],
                        "boost": 0.5
                    }
                }
            ],
            "filter": [
                {
                    "term": {
                        "type": {
                            "value": "TYPE1"
                        }
                    }
                }
            ]
        }
    },
    "size": 11,
    "from": 30
}

```

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 12, 2021, 11:30pm UTC](https://discuss.elastic.co/t/duplicate-results-when-paging/260895/2 "2021-01-12T23:30:53Z")

</div>

Is the index static, in that there are no new documents or changes coming into it

---

<div class="post-metadata">

### Author: ![noah-elastic](https://avatars.discourse-cdn.com/v4/letter/n/5e9695/32.png) [@noah-elastic](https://discuss.elastic.co/u/noah-elastic)
#### Post date: [January 13, 2021, 2:46pm UTC](https://discuss.elastic.co/t/duplicate-results-when-paging/260895/3 "2021-01-13T14:46:54Z")

</div>

> [@warkolm](#):
>
> Is the index static, in that there are no new documents or changes coming into it

No. Documents could be added to the index while we're searching. I'm not sure if any were added when I was testing. What happens if it is static vs not static?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 10, 2021, 2:46pm UTC](https://discuss.elastic.co/t/duplicate-results-when-paging/260895/4 "2021-02-10T14:46:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
