Upgrade to 1.4.2 from 1.1.1 causes spike in Disk IO

Hi there,

We just completed this past Monday an ES upgrade from 1.1.1 to 1.4.2 and a
Java upgrade from 7 to 8. No other changes. Same boxes, same specs, no new
queries or filters, no increased requests. What I see in Marvel since the
upgrade is a large spike in Disk IO:

https://lh6.googleusercontent.com/-i6OYBiqputk/VNuJC-47eII/AAAAAAAAA_g/TWxTcres-Uk/s1600/Screen%2BShot%2B2015-02-11%2Bat%2B11.37.58%2BAM.png

and a large dip in field data, filter cache and overall index memory
consumption:

https://lh5.googleusercontent.com/-2DRu7af1npc/VNuJNqsvOQI/AAAAAAAAA_o/56trKYYMiQw/s1600/Screen%2BShot%2B2015-02-11%2Bat%2B11.37.47%2BAM.png

Should I be looking at the new circuit breaker
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-fielddata.html#circuit-breaker
settings?

Any pointers are welcome!

-GS

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5e6dbf8a-4dd9-46d3-8cdb-a45c79a6b43c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

1.4.2 comes with a couple optimization which might explain the drop in
field data cache and lucene memory (most notably here is the disabling of
bloom filters by default - due to improvements in lucene they are not as
needed as before and we can save on the memory).

The IO spike might come
from https://github.com/elasticsearch/elasticsearch/pull/9683 , which is
still under heavy discussion. Can you try running _hot_threads on you
cluster and see whether you there's anything stats releated there?

On Wednesday, February 11, 2015 at 5:57:11 PM UTC+1, George Stathis wrote:

Hi there,

We just completed this past Monday an ES upgrade from 1.1.1 to 1.4.2 and a
Java upgrade from 7 to 8. No other changes. Same boxes, same specs, no new
queries or filters, no increased requests. What I see in Marvel since the
upgrade is a large spike in Disk IO:

https://lh6.googleusercontent.com/-i6OYBiqputk/VNuJC-47eII/AAAAAAAAA_g/TWxTcres-Uk/s1600/Screen%2BShot%2B2015-02-11%2Bat%2B11.37.58%2BAM.png

and a large dip in field data, filter cache and overall index memory
consumption:

https://lh5.googleusercontent.com/-2DRu7af1npc/VNuJNqsvOQI/AAAAAAAAA_o/56trKYYMiQw/s1600/Screen%2BShot%2B2015-02-11%2Bat%2B11.37.47%2BAM.png

Should I be looking at the new circuit breaker
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-fielddata.html#circuit-breaker
settings?

Any pointers are welcome!

-GS

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d9ed771b-0c8b-4e32-863b-d33b1cf94da0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.