# Sort by \_doc as fast as search\_type=scan

**URL:** <https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448>\
**Category:** Elasticsearch\
**Created:** [July 10, 2017, 10:21am UTC](https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448 "2017-07-10T10:21:17Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![weibin.wu](https://avatars.discourse-cdn.com/v4/letter/w/4491bb/32.png) [@weibin.wu](https://discuss.elastic.co/u/weibin.wu)\
**Post date:** [July 10, 2017, 10:21am UTC](https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448/1 "2017-07-10T10:21:17Z")

</div>

Hi ES:

I upgrade my cluster from 1.4 to 5.1.

However, I found search\_type=scan is deprecated. And the document recommended us to use sort["\_doc"] instead.

I tried to sort by \_doc during searching but I found the speed is 5 times slower than using \_scan in 1.4.

My use case is to extract all the document out in bulk format from one index.

Any ideas?

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [July 14, 2017, 8:41am UTC](https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448/2 "2017-07-14T08:41:41Z")

</div>

Do you have any changes to the mappings between 1.4 and 5.1, also does you 1.4 cluster and 5.1 contain the same amount of documents and indices?

Can you maybe also share the exact search request that you used for ES 1.4 and ES 5.1?

---

<div class="post-metadata">

**Author:** ![weibin.wu](https://avatars.discourse-cdn.com/v4/letter/w/4491bb/32.png) [@weibin.wu](https://discuss.elastic.co/u/weibin.wu)\
**Post date:** [July 18, 2017, 3:21pm UTC](https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448/3 "2017-07-18T15:21:40Z")

</div>

the mapping in 5.x

```
{
  "aliases": {},
 "mappings": {
"changeling-models-logling": {
  "properties": {
    "id": {
      "type": "keyword"
    },
    "klass": {
      "type": "keyword"
    },
    "modifications": {
      "type": "text"
    },
    "modified_at": {
      "type": "date"
    },
    "modified_by": {
      "fields": {
        "keyword": {
          "ignore_above": 256,
          "type": "keyword"
        }
      },
      "type": "text"
    },
    "modified_fields": {
      "type": "text"
    },
    "oid": {
      "type": "keyword"
    }
  }
}
 },
"settings": {
"index": {
  "number_of_replicas": "1",
  "number_of_shards": "5"
}
}
}

```

mapping in 1.4

```
{
"aliases": {
  
},
"mappings": {
  "changeling/models/logling": {
    "properties": {
      "id": {
        "type": "string"
      },
      "klass": {
        "type": "string"
      },
      "modifications": {
        "type": "string"
      },
      "modified_at": {
        "type": "date",
        "format": "dateOptionalTime"
      },
      "modified_by": {
        "type": "string"
      },
      "modified_fields": {
        "type": "string",
        "analyzer": "keyword"
      },
      "oid": {
        "type": "string"
      }
    }
  }
},
"settings": {
  "index": {
    "number_of_replicas": "1",
    "number_of_shards": "5",
    "refresh_interval": "60s"
  }
},
"warmers": {
  
}
}

```

The query I use to dump the data is  
POST /myteksi-changeling\_changeling\_models\_loglings\_2015\_04\_5x/\_search?\_source=true&scroll=2m&sort=\_doc%3Aasc  
{ "query": { "bool": { "filter" : [{ "range" : {"modified\_at" : {"from" :"2015-04-01","to": "2015-04-01","time\_zone":"+08:00"}}}]}}}  
POST /\_search/scroll?scroll=2m

---

<div class="post-metadata">

**Author:** ![rjernst](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rjernst/32/6363_2.png) [@rjernst](https://discuss.elastic.co/u/rjernst)\
**Post date:** [July 19, 2017, 6:59pm UTC](https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448/4 "2017-07-19T18:59:00Z")

</div>

One difference I see is the refresh interval. In 1.4 you have it set to 60 seconds, but you leave it as the default in 5.0 (which is 1 second). That may or may not cause issues (more merging in the background, so more cpu not available for searches).

But also your request specifies sort on `doc`, which it should be `_doc`. I'm confused on how this does not raise an error. Can you confirm you passed in `sort=doc` and it did not give an error for you?

---

<div class="post-metadata">

**Author:** ![weibin.wu](https://avatars.discourse-cdn.com/v4/letter/w/4491bb/32.png) [@weibin.wu](https://discuss.elastic.co/u/weibin.wu)\
**Post date:** [July 19, 2017, 10:49pm UTC](https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448/5 "2017-07-19T22:49:04Z")

</div>

I just see the document in 5.x when use scroll is  
POST /\_search?scroll=2m

But i am using  
POST /\_search/scroll?scroll=2m

Does it matter?

---

<div class="post-metadata">

**Author:** ![rjernst](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rjernst/32/6363_2.png) [@rjernst](https://discuss.elastic.co/u/rjernst)\
**Post date:** [July 20, 2017, 7:06pm UTC](https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448/6 "2017-07-20T19:06:40Z")

</div>

The latter would be used if you had an index named `scroll`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 17, 2017, 7:06pm UTC](https://discuss.elastic.co/t/sort-by--doc-as-fast-as-search-type-scan/92448/7 "2017-08-17T19:06:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.