# Scroll api or search\_after giving duplicate records
**URL:** https://discuss.elastic.co/t/scroll-api-or-search-after-giving-duplicate-records/219254
**Category:** Elasticsearch
**Created:** [February 13, 2020, 4:22pm UTC](https://discuss.elastic.co/t/scroll-api-or-search-after-giving-duplicate-records/219254 "2020-02-13T16:22:53Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@arjun\_kumar](https://discuss.elastic.co/u/arjun_kumar)
#### Post date: [February 13, 2020, 4:22pm UTC](https://discuss.elastic.co/t/scroll-api-or-search-after-giving-duplicate-records/219254/1 "2020-02-13T16:22:53Z")
Hi
I would like query huge amount of data (more than 500k), to achieve this, I used scroll API and searc\_after(with modified time)
in both the cases it's giving me the same records(duplicate) multiple times
```
{
"size": "10000",
"scroll": "100s",
"body":{
"query":{
"bool":{
"must":[
{"query_string":{"query":"*"}}
]
}
}
}
}
```
also tried with the search after
```
{
"from":0,
"size": 10000,
"query": {
"match" : {
"title" : "elasticsearch"
}
},
"search_after": [1463538857],
"sort": [
{"date": "asc"}
]
}
```
I tried with both the approaches but still getting duplicates the total records count matches with total records count but getting duped records means that some of the records are got missed
kindly help me, and thanks in advance
ES version:6.8.0
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [February 13, 2020, 4:48pm UTC](https://discuss.elastic.co/t/scroll-api-or-search-after-giving-duplicate-records/219254/2 "2020-02-13T16:48:38Z")
Hey,
so `search_after` can give duplicate entries while indexing/updating of documents is happening. Scroll search however uses a point in time snapshot. Is there any chance you can reproduce that behaviour reliably with a small dataset that you can share plus all the queries?
--Alex
---
### Author:  [@arjun\_kumar](https://discuss.elastic.co/u/arjun_kumar)
#### Post date: [February 13, 2020, 6:17pm UTC](https://discuss.elastic.co/t/scroll-api-or-search-after-giving-duplicate-records/219254/3 "2020-02-13T18:17:02Z")
Hi @spinscale
Below is the sample data and query (scroll)
```
{
"took": 8,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total":500909,
"max_score": 1.0,
"hits": [
"_index": "my_index",
"_type": "simpleindex",
"_id": "13213",
"_score": 1.0,
"_source": {
"name": 2019,
"type": "type-1",
"dept": 2,
"id": "",
"status": "deleted",
"modifiedTime": "2020-02-12T13:42:54.662Z"
}
}
}
```
and query is
```
{
"size": "10000",
"scroll": "100s",
"body":{
"query":{
"bool":{
"must":[
{"query_string":{"query":"*"}}
]
}
}
}
}
```
we are trying to connect the ES with [https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/client-usage.html(ver:15.2.0)](https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/client-usage.html(ver:15.2.0))
and the usage is
response = await client.scroll({
scrollId: result.\_scroll\_id,
scroll: "30S"
})
---
### Author:  [@arjun\_kumar](https://discuss.elastic.co/u/arjun_kumar)
#### Post date: [February 14, 2020, 3:30am UTC](https://discuss.elastic.co/t/scroll-api-or-search-after-giving-duplicate-records/219254/4 "2020-02-14T03:30:27Z")
Hi @spinscale,
can you please look into this
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [February 14, 2020, 9:05am UTC](https://discuss.elastic.co/t/scroll-api-or-search-after-giving-duplicate-records/219254/5 "2020-02-14T09:05:54Z")
Hey,
first, this is a completely volunteer driven forum, so there is no guarantee of answering, also not when pinging people directly, especially not, if they have not answered within 9h after you wrote a post. If you need support with SLAs, take a look at [Elastic subscriptions](https://www.elastic.co/subscriptions).
Second, this is not an example I can reproduce locally, so it's hard to find if there is a problem with the requests.
Can you reproduce this behaviour if you do not use the javascript client, but use the dev-tools console? if you do, can you share requests and responses that you executed?
--Alex
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 13, 2020, 9:06am UTC](https://discuss.elastic.co/t/scroll-api-or-search-after-giving-duplicate-records/219254/6 "2020-03-13T09:06:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.