PerThreadIDAndVersionLookup - thread safety

Pawel_Rog · March 15, 2015, 9:16pm

Hi,
Can anyone shortly describe why class PerThreadIDAndVersionLookup is not
thread safe and what is needed to make it thread safe? I'm wondering if it
is possible to keep only single instance of VersionLookup and make it not
stick to a thread. I see waste of big chunk of memory in JVM only because
of class PerThreadIDAndVersionLookup.

Thanks a lot for any suggestions/advises.

--
Paweł

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

jprante · March 15, 2015, 11:03pm

It is not thread safe because of the TermsEnum array, which can not be
shared between threads. By not sharing, a thread can reuse the array, which
avoids expensive reinitialization.

The utility class was introduced at

github.com/elastic/elasticsearch

Indexing: Versions.loadDocIdAndVersion should reuse TermsEnums

opened 04:36PM - 16 May 14 UTC

closed 09:37PM - 31 May 14 UTC

mikemccand

>enhancement v2.0.0-beta1 v1.3.0 release highlight

Today, every version lookup creates a new TermsEnum for each segment in the inde…x, but this is quite costly, e.g. on NIOFSDir it must clone the IO buffer, and because BlockTreeTermsReader has a lot of internal state. We'd need a ThreadLocal somewhere/somehow... I have a start at a utility class here: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene5675/lucene/test-framework/src/java/org/apache/lucene/index/PerThreadPKLookup.java maybe we can adapt/use this.

and from what I understand this replaced the previous version ID lookup by
bloom filters (which comes with a very noticeable RAM cost)

Maybe you have lots of segments?

Sometimes, ThreadLocals go crazy because of Java issues, and they are hard
to clean up. So I think if you can post some more detailed information
about what you have seen and what OS, JVM, and ES versions you use, it
would be helpful.

Jörg

On Sun, Mar 15, 2015 at 10:16 PM, Paweł Róg prog88@gmail.com wrote:

Hi,
Can anyone shortly describe why class PerThreadIDAndVersionLookup is not
thread safe and what is needed to make it thread safe? I'm wondering if it
is possible to keep only single instance of VersionLookup and make it not
stick to a thread. I see waste of big chunk of memory in JVM only because
of class PerThreadIDAndVersionLookup.

Thanks a lot for any suggestions/advises.

--
Paweł

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFaLjw58opRVQ79_fXxtzRDRoFDZz2y2qRGz1bVbAX6jg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Pawel_Rog · March 16, 2015, 1:29pm

Hi,
If only TermsEnum array is not thread safe, maybe it's worth to consider
sticking only it (or all other required arrays) to a thread?
Heap dump report showed me that a lot of space is taken by versions
NumericDocValues array.

--
Paweł

W dniu poniedziałek, 16 marca 2015 00:03:59 UTC+1 użytkownik Jörg Prante
napisał:

It is not thread safe because of the TermsEnum array, which can not be
shared between threads. By not sharing, a thread can reuse the array, which
avoids expensive reinitialization.

The utility class was introduced at

Indexing: Versions.loadDocIdAndVersion should reuse TermsEnums · Issue #6212 · elastic/elasticsearch · GitHub

and from what I understand this replaced the previous version ID lookup by
bloom filters (which comes with a very noticeable RAM cost)

Maybe you have lots of segments?

Sometimes, ThreadLocals go crazy because of Java issues, and they are hard
to clean up. So I think if you can post some more detailed information
about what you have seen and what OS, JVM, and ES versions you use, it
would be helpful.

Jörg

On Sun, Mar 15, 2015 at 10:16 PM, Paweł Róg <pro...@gmail.com
<javascript:>> wrote:

Hi,
Can anyone shortly describe why class PerThreadIDAndVersionLookup is not
thread safe and what is needed to make it thread safe? I'm wondering if it
is possible to keep only single instance of VersionLookup and make it not
stick to a thread. I see waste of big chunk of memory in JVM only because
of class PerThreadIDAndVersionLookup.

Thanks a lot for any suggestions/advises.

--
Paweł

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9af61116-2abb-4995-aa0c-86a39b046337%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jprante · March 16, 2015, 2:55pm

Sorry for being unclear, the TermsEnum array is one (the most important) of
the arrays for iteration, the other arrays are also not thread safe - you
can view all the private class variables as a thread-private cache.
NumericDocValues is the key component for retrieving the version.

Jörg

On Mon, Mar 16, 2015 at 2:29 PM, Paweł Róg prog88@gmail.com wrote:

Hi,
If only TermsEnum array is not thread safe, maybe it's worth to consider
sticking only it (or all other required arrays) to a thread?
Heap dump report showed me that a lot of space is taken by versions
NumericDocValues array.

--
Paweł

W dniu poniedziałek, 16 marca 2015 00:03:59 UTC+1 użytkownik Jörg Prante
napisał:

It is not thread safe because of the TermsEnum array, which can not be
shared between threads. By not sharing, a thread can reuse the array, which
avoids expensive reinitialization.

The utility class was introduced at

Indexing: Versions.loadDocIdAndVersion should reuse TermsEnums · Issue #6212 · elastic/elasticsearch · GitHub

and from what I understand this replaced the previous version ID lookup
by bloom filters (which comes with a very noticeable RAM cost)

Maybe you have lots of segments?

Sometimes, ThreadLocals go crazy because of Java issues, and they are
hard to clean up. So I think if you can post some more detailed information
about what you have seen and what OS, JVM, and ES versions you use, it
would be helpful.

Jörg

On Sun, Mar 15, 2015 at 10:16 PM, Paweł Róg pro...@gmail.com wrote:

Hi,
Can anyone shortly describe why class PerThreadIDAndVersionLookup is
not thread safe and what is needed to make it thread safe? I'm wondering if
it is possible to keep only single instance of VersionLookup and make it
not stick to a thread. I see waste of big chunk of memory in JVM only
because of class PerThreadIDAndVersionLookup.

Thanks a lot for any suggestions/advises.

--
Paweł

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_
oPJcs4LqyYA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAHngsdi_u_gj0PAaahB%2B8fEhsqRQ0SNr5LrFw5_oPJcs4LqyYA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9af61116-2abb-4995-aa0c-86a39b046337%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9af61116-2abb-4995-aa0c-86a39b046337%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG41vXVG5q-ayQtSGVuj8zhkUe%3DXbavRxkH%2BQYP0HD%3Dig%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.