Elsticsearch JDBC river plugin metrics

I'm trying to run a river query which fetches 10000 records scheduled at 1
minute interval. The first time it runs, metrics is 10000 rows and after a
gap of 1 minute that is (scheduled interval) metrics is 20000 rows. What
does this mean? Although the number of hits i get on querying the river
index is 10000 itself. Why do the metrics on rows keep on increasing by a
factor of 10000 each time?

[2015-01-16 16:38:03,406][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@20f4b6fe complete: river
jdbc/my_jdbc_river metrics: 10000 rows, 250.5428448620193 mean, (0.0 0.0
0.0), ingest metrics: elapsed 3 seconds, 3.37 MB bytes, 352.0 bytes avg, 1
MB/s

and also can anyone explain the below values?

metrics: 10000 rows, 250.5428448620193 mean, (0.0 0.0 0.0),
ingest metrics: elapsed 3 seconds, 3.37 MB bytes, 352.0 bytes avg, 1 MB/s.

Thank you.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a043ffe1-976d-4c07-bca6-a7ef93f14b3b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

These are diagnostic messages which have been crept into one of the
releases. Latest version has metrics logging disabled, it must be enabled
by settings.

The metrics count the number of rows fetched form the database, and prints
them at every minute. This is not the number of documents in ES.

The metrics print an average mean of the row count, so you can see that
your database sent 250 rows per second. It also counts the data volume in
bytes, and print the measure of megabytes per second, which is in
interesting number for throughput.

Jörg

On Fri, Jan 16, 2015 at 12:16 PM, 4m7u1 amtulnazneen7@gmail.com wrote:

I'm trying to run a river query which fetches 10000 records scheduled at 1
minute interval. The first time it runs, metrics is 10000 rows and after a
gap of 1 minute that is (scheduled interval) metrics is 20000 rows. What
does this mean? Although the number of hits i get on querying the river
index is 10000 itself. Why do the metrics on rows keep on increasing by a
factor of 10000 each time?

[2015-01-16 16:38:03,406][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@20f4b6fe complete: river
jdbc/my_jdbc_river metrics: 10000 rows, 250.5428448620193 mean, (0.0 0.0
0.0), ingest metrics: elapsed 3 seconds, 3.37 MB bytes, 352.0 bytes avg, 1
MB/s

and also can anyone explain the below values?

metrics: 10000 rows, 250.5428448620193 mean, (0.0 0.0 0.0),
ingest metrics: elapsed 3 seconds, 3.37 MB bytes, 352.0 bytes avg, 1 MB/s.

Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a043ffe1-976d-4c07-bca6-a7ef93f14b3b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a043ffe1-976d-4c07-bca6-a7ef93f14b3b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHbGD7YigmdgmZv%2BC4_oe4m32iCg4-eHUKXf4S0urpXcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thank you so much Jörg :slight_smile: !

On Friday, January 16, 2015 at 6:08:49 PM UTC+5:30, Jörg Prante wrote:

These are diagnostic messages which have been crept into one of the
releases. Latest version has metrics logging disabled, it must be enabled
by settings.

The metrics count the number of rows fetched form the database, and prints
them at every minute. This is not the number of documents in ES.

The metrics print an average mean of the row count, so you can see that
your database sent 250 rows per second. It also counts the data volume in
bytes, and print the measure of megabytes per second, which is in
interesting number for throughput.

Jörg

On Fri, Jan 16, 2015 at 12:16 PM, 4m7u1 <amtuln...@gmail.com <javascript:>

wrote:

I'm trying to run a river query which fetches 10000 records scheduled at
1 minute interval. The first time it runs, metrics is 10000 rows and after
a gap of 1 minute that is (scheduled interval) metrics is 20000 rows. What
does this mean? Although the number of hits i get on querying the river
index is 10000 itself. Why do the metrics on rows keep on increasing by a
factor of 10000 each time?

[2015-01-16 16:38:03,406][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@20f4b6fe complete: river
jdbc/my_jdbc_river metrics: 10000 rows, 250.5428448620193 mean, (0.0 0.0
0.0), ingest metrics: elapsed 3 seconds, 3.37 MB bytes, 352.0 bytes avg, 1
MB/s

and also can anyone explain the below values?

metrics: 10000 rows, 250.5428448620193 mean, (0.0 0.0 0.0),
ingest metrics: elapsed 3 seconds, 3.37 MB bytes, 352.0 bytes avg, 1 MB/s.

Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a043ffe1-976d-4c07-bca6-a7ef93f14b3b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a043ffe1-976d-4c07-bca6-a7ef93f14b3b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9338605d-5b9d-4af2-8b62-625f64c7509a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi,
I've got a small doubt again.

[2015-01-19 15:06:08,410][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@1cede282 complete: river
jdbc/my_test_river metrics: 340000 rows, 170.35844075426294 mean,
(147.82076731011736 30.98274869393544 10.409690875802436), ingest metrics:
elapsed 8 seconds, 57.57 MB bytes, 177.0 bytes avg, 6.848 MB/s

So here,
Average no of rows picked per second is 170 and what do the values in the
brackets mean (147.82076731011736 30.98274869393544 10.409690875802436).
and also ingest metrics: elapsed 8 seconds mean that the total time taken
in the corresponding poll is 8 seconds?

Thank you.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4c9f209b-cefb-4709-a2b9-a98f971eecb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

The numbers in brackets mean the average numbers over the last 1 / 5 / 15
minutes. This useful for a long running indexer.

The "elapsed" time is the time since this metric counter is active.

Jörg

On Mon, Jan 19, 2015 at 10:41 AM, 4m7u1 amtulnazneen7@gmail.com wrote:

Hi,
I've got a small doubt again.

[2015-01-19 15:06:08,410][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@1cede282 complete: river
jdbc/my_test_river metrics: 340000 rows, 170.35844075426294 mean,
(147.82076731011736 30.98274869393544 10.409690875802436), ingest metrics:
elapsed 8 seconds, 57.57 MB bytes, 177.0 bytes avg, 6.848 MB/s

So here,
Average no of rows picked per second is 170 and what do the values in the
brackets mean (147.82076731011736 30.98274869393544 10.409690875802436).
and also ingest metrics: elapsed 8 seconds mean that the total time taken
in the corresponding poll is 8 seconds?

Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4c9f209b-cefb-4709-a2b9-a98f971eecb9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4c9f209b-cefb-4709-a2b9-a98f971eecb9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH7GxfsFveCTCdk77JZ_sUySmtn33Peerk2UrRYV%2BDNuw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Okay. Thanks a lot. Is there any other way in which I can see the metrics.
If I have indexed my db with 20000 records in Elasticsearch, for that I can
see the metrics as above. But say that, I have made one new
insert/update/delete into my db and that is picked by elasticsearch, is
there any way in which I can see the time taken for that one particular
insert/update/delete?

On Monday, January 19, 2015 at 3:36:47 PM UTC+5:30, Jörg Prante wrote:

The numbers in brackets mean the average numbers over the last 1 / 5 / 15
minutes. This useful for a long running indexer.

The "elapsed" time is the time since this metric counter is active.

Jörg

On Mon, Jan 19, 2015 at 10:41 AM, 4m7u1 <amtuln...@gmail.com <javascript:>

wrote:

Hi,
I've got a small doubt again.

[2015-01-19 15:06:08,410][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@1cede282 complete: river
jdbc/my_test_river metrics: 340000 rows, 170.35844075426294 mean,
(147.82076731011736 30.98274869393544 10.409690875802436), ingest metrics:
elapsed 8 seconds, 57.57 MB bytes, 177.0 bytes avg, 6.848 MB/s

So here,
Average no of rows picked per second is 170 and what do the values in
the brackets mean (147.82076731011736 30.98274869393544
10.409690875802436).
and also ingest metrics: elapsed 8 seconds mean that the total time taken
in the corresponding poll is 8 seconds?

Thank you.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4c9f209b-cefb-4709-a2b9-a98f971eecb9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4c9f209b-cefb-4709-a2b9-a98f971eecb9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/139a365e-aca7-45e3-b1d7-7bd3a08dfbda%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

As said, insert/update/delete in DB are not visible to the JDBC plugin and
they can not be detected by the JDBC plugin. The metric is for the SQL
select statement the plugin is using. If you mean insert/update/delete in
ES, there is no such thing, there are indexing operations performed in
bulk, which can be monitored by ES monitoring plugins or ES metrics.

Jörg

On Mon, Jan 19, 2015 at 11:55 AM, 4m7u1 amtulnazneen7@gmail.com wrote:

Okay. Thanks a lot. Is there any other way in which I can see the metrics.
If I have indexed my db with 20000 records in Elasticsearch, for that I can
see the metrics as above. But say that, I have made one new
insert/update/delete into my db and that is picked by elasticsearch, is
there any way in which I can see the time taken for that one particular
insert/update/delete?

On Monday, January 19, 2015 at 3:36:47 PM UTC+5:30, Jörg Prante wrote:

The numbers in brackets mean the average numbers over the last 1 / 5 / 15
minutes. This useful for a long running indexer.

The "elapsed" time is the time since this metric counter is active.

Jörg

On Mon, Jan 19, 2015 at 10:41 AM, 4m7u1 amtuln...@gmail.com wrote:

Hi,
I've got a small doubt again.

[2015-01-19 15:06:08,410][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@1cede282 complete: river
jdbc/my_test_river metrics: 340000 rows, 170.35844075426294 mean,
(147.82076731011736 30.98274869393544 10.409690875802436), ingest metrics:
elapsed 8 seconds, 57.57 MB bytes, 177.0 bytes avg, 6.848 MB/s

So here,
Average no of rows picked per second is 170 and what do the values in
the brackets mean (147.82076731011736 30.98274869393544
10.409690875802436).
and also ingest metrics: elapsed 8 seconds mean that the total time
taken in the corresponding poll is 8 seconds?

Thank you.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/4c9f209b-cefb-4709-a2b9-a98f971eecb9%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4c9f209b-cefb-4709-a2b9-a98f971eecb9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/139a365e-aca7-45e3-b1d7-7bd3a08dfbda%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/139a365e-aca7-45e3-b1d7-7bd3a08dfbda%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH8bvDfBAxsP0vi3GKpzDfTs1_dpH%3D62br9SaJYUBHwsA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Got it. Thanks !

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a1f56595-9ab1-4e89-97d4-6e257cbb3042%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.