# Duplicate results
**URL:** https://discuss.elastic.co/t/duplicate-results/295808
**Category:** Elasticsearch
**Created:** [January 31, 2022, 8:45am UTC](https://discuss.elastic.co/t/duplicate-results/295808 "2022-01-31T08:45:42Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@motoki](https://discuss.elastic.co/u/motoki)
#### Post date: [January 31, 2022, 8:45am UTC](https://discuss.elastic.co/t/duplicate-results/295808/1 "2022-01-31T08:45:42Z")
Elasticsearch.... . 7.4
Status... Duplicate results were seen in pagination search results when there were multiple data nodes, regardless of whether there was a dedicated master or not (the same was true when specifying a shard ID or custom string in preference). When index was restored from a snapshot, no duplication was found.
Questions
(1) I would like to configure multiple data nodes to prevent duplication during pagination search and to ensure availability. How can I do this? (Solution by configuration, system flow, etc.)
(2) At the time of verification, there was no duplication when index was restored from snapshot, but is this the case in the specification? (If so, I think it can be achieved by separating them in the read/write index.)
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 31, 2022, 9:38am UTC](https://discuss.elastic.co/t/duplicate-results/295808/2 "2022-01-31T09:38:22Z")
Welcome to our community! 😃
7.4 is [EOL](https://www.elastic.co/support/eol) and no longer supported, please upgrade. 7.16 is latest and 8.0 is currently in alpha 🙂
That said, you'd need to share your query and it's results, and document samples, to allow us to comment further.
---
### Author:  [@motoki](https://discuss.elastic.co/u/motoki)
#### Post date: [February 11, 2022, 7:32am UTC](https://discuss.elastic.co/t/duplicate-results/295808/3 "2022-02-11T07:32:54Z")
Thanks for the reply.
We will consider the upgrade separately.
The document sample, query and result samples are shown below.
However, the key name is replaced with "v\_", the string with "xxxxx", and the number with "yyyyy".
Document Sample
```auto
{
"_index": "xxxxx",
"_type": "_doc",
"_id": "xxxxx",
"_version": 6,
"_seq_no": 1246083,
"_primary_term": 2,
"found": true,
"_source": {
"v_1": "xxxxx",
"v_8": yyyyy,
"v_3": "xxxxx",
"v_10": yyyyy,
"v_11": yyyyy,
"f_12": true,
"v_7": "xxxxx",
"v_2": "xxxxx",
"v_5": "xxxxx",
"v_4": "xxxxx",
"v_9": yyyyy,
"v_6": "xxxxx",
"v_13": {
"v_14": null,
"v_15": null
},
}
}
```
Request Sample
```auto
{
"track_total_hits": True,
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "xxxxxx",
"type": "phrase",
"fields": [
"v_1^1.5",
"v_2",
"v_3",
"v_4",
"v_5",
"v_6^0.9",
"v_7"
],
"analyzer": "ja_analyzer",
"slop": 2
}
}
],
"filter": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"range": {
"v_8": {
"gt": 0
}
}
},
{
"match": {
"v_8": -1
}
}
]
}
},
{
"bool": {
"should": [
{
"range": {
"v_9": {
"gt": 0
}
}
},
{
"match": {
"v_9": -1
}
}
]
}
},
{
"bool": {
"should": [
{
"range": {
"v_10": {
"gte": 0
}
}
},
{
"match": {
"v_10": -1
}
}
]
}
},
{
"bool": {
"should": [
{
"range": {
"v_11": {
"gte": 0
}
}
},
{
"match": {
"v_11": -1
}
}
]
}
},
{
"term": {
"v_12": True
}
},
{
"bool": {
"should": []
}
},
{
"bool": {
"should": []
}
},
{
"bool": {
"should": []
}
}
]
}
}
}
},
"from": 0,
"size": per,
"highlight": {
"pre_tags": [
""
],
"post_tags": [
""
],
"fields": {
"v_6": {
"number_of_fragments": 1,
"fragment_size": 150
},
"v_7": {
"number_of_fragments": 1,
"fragment_size": 150
},
"v_5": {
"number_of_fragments": 1,
"fragment_size": 150
},
"v_4": {
"number_of_fragments": 1,
"fragment_size": 150
},
"v_2": {
"number_of_fragments": 1,
"fragment_size": 150
},
"v_3": {
"number_of_fragments": 1,
"fragment_size": 150
}
}
},
"_source": {
"excludes": [
"v_6",
"v_13"
]
}
}
```
Result Sample
```auto
{
"took": 207,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 8557,
"relation": "eq"
},
"max_score": 10.259903,
"hits": [
{
"_index": "xxxxx",
"_type": "_doc",
"_id": "xxxxx",
"_score": 10.259903,
"_source": {
"v_8": yyyyy,
"v_7": "xxxxx",
"v_5": "xxxxx",
"v_3": "xxxxx",
"v_1": "xxxxx",
"v_4": "xxxxx",
"v_9": yyyyy,
"v_10": yyyyy,
"v_12": true,
"v_11": yyyyy,
"v_2": "xxxx",
},
"highlight": {
"v_5": [
"xxxxx"
],
"v_6": [
"xxxxx"
],
"v_3": [
"xxxxx"
],
"v_2": [
"xxxxx"
]
}
},
.....
]
}
}
```
---
### Author:  [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)
#### Post date: [February 12, 2022, 1:49pm UTC](https://discuss.elastic.co/t/duplicate-results/295808/4 "2022-02-12T13:49:26Z")
I suppose you need to share query for pagination.
---
### Author:  [@motoki](https://discuss.elastic.co/u/motoki)
#### Post date: [February 14, 2022, 7:51am UTC](https://discuss.elastic.co/t/duplicate-results/295808/5 "2022-02-14T07:51:02Z")
> [@Tomo\_M](#):
>
> I suppose you need to share query for pagination.
I get it in python as follows.
```auto
import urllib.request
import json
import copy
per = 20
query={....}
headers = {
'Content-Type': 'application/json',
}
for i in range(0, 50):
q = copy.deepcopy(query)
q['from'] = i * per
json_data = json.dumps(q).encode('utf-8')
req = urllib.request.Request(
url,
data=json_data,
headers=headers,
method='GET'
)
res = urllib.request.urlopen(req)
```
---
### Author:  [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)
#### Post date: [February 14, 2022, 8:49am UTC](https://discuss.elastic.co/t/duplicate-results/295808/6 "2022-02-14T08:49:26Z")
Your query doesn't store status of the index, and the sorted order recreated on the every queries.
When some refresh occurs between queries, the sorted order changes, as explained in the first paragraph of [Point in time API](https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html#point-in-time-api). Point in time API is the uptodate way to cope with the problem.
Although it is already not recommended function in 8.0, [Scroll](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/search-request-body.html#request-body-search-scroll) may help you.
---
### Author:  [@motoki](https://discuss.elastic.co/u/motoki)
#### Post date: [February 15, 2022, 2:21am UTC](https://discuss.elastic.co/t/duplicate-results/295808/7 "2022-02-15T02:21:42Z")
Sorry, this time the situation is as follows.
(1) For a single node... No duplication
(2)In the case of multiple nodes
(i)When index is restored... No duplication occurs.
(ii)When an index is created... Duplication will occur.
In the case of (ii), there is no update of index when searching.
What we want to know is
(a) Why does this happen (difference in the number of nodes, difference in restore/create)?
(b) How to avoid duplication when retrieving multiple nodes and pagination.
Of course, we understand that the version we are using is old, so we are considering upgrading separately.
---
### Author:  [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)
#### Post date: [February 15, 2022, 3:10am UTC](https://discuss.elastic.co/t/duplicate-results/295808/8 "2022-02-15T03:10:00Z")
(a) I'm not sure but I suppose getting not duplicated results by `from` query was only by chance and it is not to be expected.
(b) explained in the previous post. Try Scroll.
---
### Author:  [@motoki](https://discuss.elastic.co/u/motoki)
#### Post date: [February 17, 2022, 8:40am UTC](https://discuss.elastic.co/t/duplicate-results/295808/9 "2022-02-17T08:40:57Z")
Sample documents and other information can be found above.
I've also included what I'd like to know about the current situation.
If you have any advice, it would be greatly appreciated.
---
### Author:  [@motoki](https://discuss.elastic.co/u/motoki)
#### Post date: [February 25, 2022, 6:59am UTC](https://discuss.elastic.co/t/duplicate-results/295808/10 "2022-02-25T06:59:26Z")
I would be grateful for any advice anyone can give me.
As a motivation for this project, a search in pagination is a must.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 25, 2022, 7:12am UTC](https://discuss.elastic.co/t/duplicate-results/295808/11 "2022-02-25T07:12:20Z")
You should look at [Point in time API | Elasticsearch Guide [8.0] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html#point-in-time-api) for consistent pagination IMHO.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 25, 2022, 7:13am UTC](https://discuss.elastic.co/t/duplicate-results/295808/12 "2022-03-25T07:13:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.