Jdbc river keeping connection open

Hi all,

we have some JUnit based integration tests in which we basically setup a
clean system, create a river and do some index operations like
inserting/updating documents. We are not using ElasticSearchIntegrationTest
but are running our tests against a dedicated cluster.

In the setup which is performed before every test run we do the following:

  1. remove jdbc river from last test run
  2. remove index from last test run
  3. truncate all tables in database

We are facing the issue that step 3 sometimes blocks because there is an
open connection from the river.

The deletion is performed by this code:
DeleteMappingResponse response = client.admin().indices()
.prepareDeleteMapping("_river")
.setType(rivername)
.execute()
.actionGet();

return response.isAcknowledged();

Is this a known issue or do we have some misunderstanding of how to deal
with river deletion?

May it be the case that the table truncation is performed when the river
deletion isn't fully completed? If so, is there a way to wait until river
deletion completed?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/00a6d81c-5f72-4c16-a814-dcb771d3111b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Are you sure the deletion of a (non-existent) mapping "_river" makes sense?

What connection is open? Database connection?

Jörg
Am 16.02.2015 12:29 schrieb "Abid Hussain" hussain@novacom.mygbiz.com:

Hi all,

we have some JUnit based integration tests in which we basically setup a
clean system, create a river and do some index operations like
inserting/updating documents. We are not using ElasticSearchIntegrationTest
but are running our tests against a dedicated cluster.

In the setup which is performed before every test run we do the following:

  1. remove jdbc river from last test run
  2. remove index from last test run
  3. truncate all tables in database

We are facing the issue that step 3 sometimes blocks because there is an
open connection from the river.

The deletion is performed by this code:
DeleteMappingResponse response = client.admin().indices()
.prepareDeleteMapping("_river")
.setType(rivername)
.execute()
.actionGet();

return response.isAcknowledged();

Is this a known issue or do we have some misunderstanding of how to deal
with river deletion?

May it be the case that the table truncation is performed when the river
deletion isn't fully completed? If so, is there a way to wait until river
deletion completed?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/00a6d81c-5f72-4c16-a814-dcb771d3111b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/00a6d81c-5f72-4c16-a814-dcb771d3111b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHs1Yr2hGRv-Hg1Q0ufCEZY720boZwoU%2BXAeudx-nJ1gQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Are you sure the deletion of a (non-existent) mapping "_river" makes sense?

The given code results in river deletion. If there is a better way to do
so, I would be happy if you could tell me.

What connection is open? Database connection?

Yes, a database connection used by river remains open.

Regards,

Abid

Jörg
Am 16.02.2015 12:29 schrieb "Abid Hussain" <hus...@novacom.mygbiz.com
<javascript:>>:

Hi all,

we have some JUnit based integration tests in which we basically setup a
clean system, create a river and do some index operations like
inserting/updating documents. We are not using ElasticSearchIntegrationTest
but are running our tests against a dedicated cluster.

In the setup which is performed before every test run we do the following:

  1. remove jdbc river from last test run
  2. remove index from last test run
  3. truncate all tables in database

We are facing the issue that step 3 sometimes blocks because there is an
open connection from the river.

The deletion is performed by this code:
DeleteMappingResponse response = client.admin().indices()
.prepareDeleteMapping("_river")
.setType(rivername)
.execute()
.actionGet();

return response.isAcknowledged();

Is this a known issue or do we have some misunderstanding of how to deal
with river deletion?

May it be the case that the table truncation is performed when the river
deletion isn't fully completed? If so, is there a way to wait until river
deletion completed?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/00a6d81c-5f72-4c16-a814-dcb771d3111b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/00a6d81c-5f72-4c16-a814-dcb771d3111b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7108a8e1-8695-40e2-a269-4d3b32773dee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I noticed in the elasticsearch logs:
[2015-02-16 15:25:20,398][INFO ][river.jdbc.RiverMetrics ] pipeline
org.xbib.elasticsearch.plugin.jdbc.RiverPipeline@25042719 complete: river
jdbc/ctv4ch_junit_abid_interview metrics: 1 rows, 0.23379861173437763 mean,
(0.0 0.0 0.0), ingest metrics: elapsed 4 seconds, 0.0 bytes bytes, 0.0
bytes avg, 0 MB/s
[2015-02-16 15:25:20,398][ERROR][river.jdbc.BulkNodeClient] after bulk [1]
error
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1301)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:317)
at
org.elasticsearch.action.bulk.BulkProcessor.execute(BulkProcessor.java:325)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.xbib.elasticsearch.plugin.jdbc.client.BulkProcessorHelper.flush(BulkProcessorHelper.java:43)

This seems to occur when the river, which has just been created, is removed
again by the next test run. So it seems the an InterruptedException results
in an open connection remaining.

I did the following: before deletion of river; I wait until the previously
created one did finish (via _river/jdbc//_state. Then I delete.

This seems to work.

Regards,

Abid

Am Montag, 16. Februar 2015 14:43:37 UTC+1 schrieb Abid Hussain:

Are you sure the deletion of a (non-existent) mapping "_river" makes sense?

The given code results in river deletion. If there is a better way to do
so, I would be happy if you could tell me.

What connection is open? Database connection?

Yes, a database connection used by river remains open.

Regards,

Abid

Jörg
Am 16.02.2015 12:29 schrieb "Abid Hussain" hus...@novacom.mygbiz.com:

Hi all,

we have some JUnit based integration tests in which we basically setup a
clean system, create a river and do some index operations like
inserting/updating documents. We are not using ElasticSearchIntegrationTest
but are running our tests against a dedicated cluster.

In the setup which is performed before every test run we do the
following:

  1. remove jdbc river from last test run
  2. remove index from last test run
  3. truncate all tables in database

We are facing the issue that step 3 sometimes blocks because there is an
open connection from the river.

The deletion is performed by this code:
DeleteMappingResponse response = client.admin().indices()
.prepareDeleteMapping("_river")
.setType(rivername)
.execute()
.actionGet();

return response.isAcknowledged();

Is this a known issue or do we have some misunderstanding of how to deal
with river deletion?

May it be the case that the table truncation is performed when the river
deletion isn't fully completed? If so, is there a way to wait until river
deletion completed?

Regards,

Abid

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/00a6d81c-5f72-4c16-a814-dcb771d3111b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/00a6d81c-5f72-4c16-a814-dcb771d3111b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3885a810-218e-44a9-b31d-2fd51a37c86e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.