What does it mean when refresh rate is high?

javadevmtl · March 26, 2015, 6:25pm

Using 1.4.3

So I have a nice "beefy" cluster 4 nodes of 32 cores, 128GB RAM, 5TB RAID 0
(Using Sandisk 960GB Extreme Pro) for each machine.

I am indexing about 4000 documents per second at an average of about 800
bytes per document. At the same time as indexing I'm running queries.

Looking at Elastic HQ numbers.

Indexing - Index:0.28ms0.32ms0.3ms0.32msIndexing - Delete:0ms0ms0ms0msSearch

Query:29.23ms29.36ms24.46ms36.63msSearch - Fetch:0.25ms0.24ms0.25ms0.21msGet
Total:0.67ms0.46ms0ms0.48msGet - Exists:1.19ms0.65ms0ms0.48msGet -
Missing:0ms0.03ms0ms0msRefresh:25.32ms24.86ms24.5ms24.81msFlush:104.14ms
90.45ms111.14ms84.63msNo matter what test I'v done or machine configuration
the refresh rate has always been red... What does it mean and does it
matter?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2da09094-f573-42e9-963f-7f9afa880de7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

warkolm · March 28, 2015, 5:39am

If you mouse over the field you can see what the value means.

On 27 March 2015 at 05:25, John Smith java.dev.mtl@gmail.com wrote:

Using 1.4.3

So I have a nice "beefy" cluster 4 nodes of 32 cores, 128GB RAM, 5TB RAID
0 (Using Sandisk 960GB Extreme Pro) for each machine.

I am indexing about 4000 documents per second at an average of about 800
bytes per document. At the same time as indexing I'm running queries.

Looking at Elastic HQ numbers.

Indexing - Index:0.28ms0.32ms0.3ms0.32msIndexing - Delete:0ms0ms0ms0msSearch

Query:29.23ms29.36ms24.46ms36.63msSearch - Fetch:0.25ms0.24ms0.25ms
0.21msGet - Total:0.67ms0.46ms0ms0.48msGet - Exists:1.19ms0.65ms0ms0.48msGet

Missing:0ms0.03ms0ms0msRefresh:25.32ms24.86ms24.5ms24.81msFlush:104.14ms
90.45ms111.14ms84.63msNo matter what test I'v done or machine
configuration the refresh rate has always been red... What does it mean and
does it matter?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2da09094-f573-42e9-963f-7f9afa880de7%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2da09094-f573-42e9-963f-7f9afa880de7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X--8N2CaBD_HgRsq7sMkOOV27tRCT8QYPZ1ya0t3_OMqQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

javadevmtl · March 30, 2015, 3:09pm

Thanks the description is just very generic "High values indicate slow I/O"

Wouldn't this mean that other I/O related tasks would also show as red.

Also given the fact that I'm running San Disk 960GB Extreme Pro in RAID 0

Performance**Seq. Read (up to)550 MB/s550 MB/s550 MB/s Seq. Write (up to)520
MB/s515 MB/s515 MB/s Rnd. Read (up to)100K IOPS100K IOPS100K IOPS Rnd.
Write (up to)90K IOPS90K IOPS90K IOPS

I also gave my indexing stats above. So why would refresh be red and not
the rest if IO is the problem.

I would figure the disk configuration would be more then adequate?

On Saturday, 28 March 2015 01:40:16 UTC-4, Mark Walkom wrote:

If you mouse over the field you can see what the value means.

On 27 March 2015 at 05:25, John Smith <java.d...@gmail.com <javascript:>>
wrote:

Using 1.4.3

So I have a nice "beefy" cluster 4 nodes of 32 cores, 128GB RAM, 5TB RAID
0 (Using Sandisk 960GB Extreme Pro) for each machine.

I am indexing about 4000 documents per second at an average of about 800
bytes per document. At the same time as indexing I'm running queries.

Looking at Elastic HQ numbers.

Indexing - Index:0.28ms0.32ms0.3ms0.32msIndexing - Delete:0ms0ms0ms0msSearch

Query:29.23ms29.36ms24.46ms36.63msSearch - Fetch:0.25ms0.24ms0.25ms
0.21msGet - Total:0.67ms0.46ms0ms0.48msGet - Exists:1.19ms0.65ms0ms0.48msGet

Missing:0ms0.03ms0ms0msRefresh:25.32ms24.86ms24.5ms24.81msFlush:
104.14ms90.45ms111.14ms84.63msNo matter what test I'v done or machine
configuration the refresh rate has always been red... What does it mean and
does it matter?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2da09094-f573-42e9-963f-7f9afa880de7%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2da09094-f573-42e9-963f-7f9afa880de7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/79874405-d44f-43c8-ba1b-c07d61838b59%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

warkolm · March 31, 2015, 2:31am

You'd have to ask the author why they have calculated it this way.

On 31 March 2015 at 02:09, John Smith java.dev.mtl@gmail.com wrote:

Thanks the description is just very generic "High values indicate slow I/O"

Wouldn't this mean that other I/O related tasks would also show as red.

Also given the fact that I'm running San Disk 960GB Extreme Pro in RAID 0

Performance**Seq. Read (up to)550 MB/s550 MB/s550 MB/s Seq. Write (up to)520
MB/s515 MB/s515 MB/s Rnd. Read (up to)100K IOPS100K IOPS100K IOPS Rnd.
Write (up to)90K IOPS90K IOPS90K IOPS

I also gave my indexing stats above. So why would refresh be red and not
the rest if IO is the problem.

I would figure the disk configuration would be more then adequate?

On Saturday, 28 March 2015 01:40:16 UTC-4, Mark Walkom wrote:

If you mouse over the field you can see what the value means.

On 27 March 2015 at 05:25, John Smith java.d...@gmail.com wrote:

Using 1.4.3

So I have a nice "beefy" cluster 4 nodes of 32 cores, 128GB RAM, 5TB
RAID 0 (Using Sandisk 960GB Extreme Pro) for each machine.

I am indexing about 4000 documents per second at an average of about 800
bytes per document. At the same time as indexing I'm running queries.

Looking at Elastic HQ numbers.

Indexing - Index:0.28ms0.32ms0.3ms0.32msIndexing - Delete:0ms0ms0ms0msSearch

Query:29.23ms29.36ms24.46ms36.63msSearch - Fetch:0.25ms0.24ms0.25ms
0.21msGet - Total:0.67ms0.46ms0ms0.48msGet - Exists:1.19ms0.65ms0ms
0.48msGet - Missing:0ms0.03ms0ms0msRefresh:25.32ms24.86ms24.5ms24.81ms
Flush:104.14ms90.45ms111.14ms84.63msNo matter what test I'v done or
machine configuration the refresh rate has always been red... What does it
mean and does it matter?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/2da09094-f573-42e9-963f-7f9afa880de7%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/2da09094-f573-42e9-963f-7f9afa880de7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/79874405-d44f-43c8-ba1b-c07d61838b59%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/79874405-d44f-43c8-ba1b-c07d61838b59%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9pNb0hbAaYT7RgY2Va7ghDnenzg%3Db5eSBWuhsrieq7gA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.