Indexing, transaction and isolation

Elastic Stack Elasticsearch
1

Hi there all!

I have an embedded ES running in a Spring backed application.

I have synchronized ES with the TransactionManager. Updates and
deletes are logged into a TransactionSynchronization context. If the
transaction commits successfully, the operations are replayed to ES.
Works great but...

We are trying to use ES to satisfy all of our search needs, not just
for client facing queries. Thus, our service layer operations would
like to rely as much as possible on ES to do searching but some
usecases can't. As soon as you need to search for something that was
created or updated within the current transaction, ES will only see
these changes after the transaction actually commits.

Question is: I know some work was done around this in Compass. Lucene
also has some basic transactional support. How would you approach
implementing 2PC in ES?

Another bit of info, we are using MySQL, Hibernate and Spring.

Thanks for your comments.

Remy

2

There isn't support for that, and its not slated to be done, especially since there isn't really a good way to do that in an acceptable perfomant manner.

On Friday, July 15, 2011 at 8:57 PM, Spring Ninja wrote:

Hi there all!

I have an embedded ES running in a Spring backed application.

I have synchronized ES with the TransactionManager. Updates and
deletes are logged into a TransactionSynchronization context. If the
transaction commits successfully, the operations are replayed to ES.
Works great but...

We are trying to use ES to satisfy all of our search needs, not just
for client facing queries. Thus, our service layer operations would
like to rely as much as possible on ES to do searching but some
usecases can't. As soon as you need to search for something that was
created or updated within the current transaction, ES will only see
these changes after the transaction actually commits.

Question is: I know some work was done around this in Compass. Lucene
also has some basic transactional support. How would you approach
implementing 2PC in ES?

Another bit of info, we are using MySQL, Hibernate and Spring.

Thanks for your comments.

Remy

3

Makes sense in a distributed engine like ES.

We'll fallback to the database for queries that need to see new data
from the current transaction.

Thanks!

Remy