# Scroll starts fast but finishes slow

**URL:** <https://discuss.elastic.co/t/scroll-starts-fast-but-finishes-slow/32787>\
**Category:** Elasticsearch\
**Created:** [October 22, 2015, 4:24pm UTC](https://discuss.elastic.co/t/scroll-starts-fast-but-finishes-slow/32787 "2015-10-22T16:24:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [October 22, 2015, 4:24pm UTC](https://discuss.elastic.co/t/scroll-starts-fast-but-finishes-slow/32787/1 "2015-10-22T16:24:16Z")

</div>

Hi running 1.7.3

I have 4 cluster node with 32 cores per node and 30GB per node and SSD drives. I have an index that has 30,000,000 records sharded with 32 shards, so 8 shards per machine.

I then run a sorted search with fields.

I set the scroll size to 100 (this doesn't seem matter to mater, tested various sizes...). Each iteration of the scroll it gets slower and slower.

So first few scrolls it runs at "decent" speed and then after that you can just see it get slower and slower...

Any thoughts on this?

Thanks

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [October 22, 2015, 6:07pm UTC](https://discuss.elastic.co/t/scroll-starts-fast-but-finishes-slow/32787/2 "2015-10-22T18:07:30Z")

</div>

Ok I have pinpointed the bottleneck.

It seems that when specifying fields it's 300 of orders magnitude slower then just getting back the full document. I'm pretty sure this used to be faster in 1.5,x

Here is the code.

[http://pastebin.com/aA2dVgbE](http://pastebin.com/aA2dVgbE)

The same report just writing the source doc line by line takes under a minute, while doing it by field it takes 30 minutes.

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [October 22, 2015, 6:16pm UTC](https://discuss.elastic.co/t/scroll-starts-fast-but-finishes-slow/32787/3 "2015-10-22T18:16:43Z")

</div>

Ok, so it was me building the header. I moved the header loop inside the if block checking to see if the header was built, and performance is back where's it supposed to be... Still don't see how that could make it slower?

This is the good way:

```
if(!headerWritten) {
	for(Entry<String, SearchHitField> e : hit.fields().entrySet()){
		header += "\"" + e.getKey() + "\",";
	}
	header += "\r\n";
	//System.out.println(header);
	fop.write(header.getBytes());
	headerWritten = true;
}

for(Entry<String, SearchHitField> e : hit.fields().entrySet()){
	line += "\"" + e.getValue().value().toString() + "\",";
}

line += "\r\n";

```

Is it because it has to go all the way to the top of the results to pull the header all the time?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:43pm UTC](https://discuss.elastic.co/t/scroll-starts-fast-but-finishes-slow/32787/4 "2017-07-05T23:43:08Z")

</div>

Closed on July 5, 2017, 11:43pm UTC