I am in the midst of tuning my ElasticSearch cluster. One major issue is
memory and some of the main culprits are facets over float values. To be
more precise, lots of distinct range facets over fields that contain prices
(11.99, 2139.00, 0.01, etc.). These ranges are combined on the client side
and mimic dynamic ranges.
Both ints and floats are 4 bytes. One common practice is to never use
floating points for currency due to rounding errors, but there are no
arithmetic operations performed on these values. Plus, I don't want to
convert legacy code. Is there any performance to be had from switching
values from floats to ints and performing divide by 100.0 on the client
side? I guess this question mainly pertains to Lucene since it deals with
the internal caches. The results can also be sorted on these values.
I don't think there is a significant difference but you should go for your
own benchmark and get some results until you're satisfied. If it's crazy
fast with floats and crazy faster with ints, it's still crazy fast
Cheers,
Jörg
On Thursday, November 8, 2012 5:02:31 PM UTC+1, Ivan Brusic wrote:
I am in the midst of tuning my Elasticsearch cluster. One major issue is
memory and some of the main culprits are facets over float values. To be
more precise, lots of distinct range facets over fields that contain prices
(11.99, 2139.00, 0.01, etc.). These ranges are combined on the client side
and mimic dynamic ranges.
Both ints and floats are 4 bytes. One common practice is to never use
floating points for currency due to rounding errors, but there are no
arithmetic operations performed on these values. Plus, I don't want to
convert legacy code. Is there any performance to be had from switching
values from floats to ints and performing divide by 100.0 on the client
side? I guess this question mainly pertains to Lucene since it deals with
the internal caches. The results can also be sorted on these values.
On Friday, November 9, 2012 2:11:32 AM UTC+1, Jörg Prante wrote:
Hi Ivan,
I don't think there is a significant difference but you should go for your
own benchmark and get some results until you're satisfied. If it's crazy
fast with floats and crazy faster with ints, it's still crazy fast
If it's crazy fast with floats and crazy faster with ints, it's still crazy
fast <== where did you get that from
Cheers,
Jörg
On Thursday, November 8, 2012 5:02:31 PM UTC+1, Ivan Brusic wrote:
I am in the midst of tuning my Elasticsearch cluster. One major issue is
memory and some of the main culprits are facets over float values. To be
more precise, lots of distinct range facets over fields that contain prices
(11.99, 2139.00, 0.01, etc.). These ranges are combined on the client side
and mimic dynamic ranges.
Both ints and floats are 4 bytes. One common practice is to never use
floating points for currency due to rounding errors, but there are no
arithmetic operations performed on these values. Plus, I don't want to
convert legacy code. Is there any performance to be had from switching
values from floats to ints and performing divide by 100.0 on the client
side? I guess this question mainly pertains to Lucene since it deals with
the internal caches. The results can also be sorted on these values.
I'd figured as much, especially in terms of memory, which is my primary
bottleneck. Code changes would be too many to do a quick benchmark. Wish I
was near crazy fast!
I don't think there is a significant difference but you should go for your
own benchmark and get some results until you're satisfied. If it's crazy
fast with floats and crazy faster with ints, it's still crazy fast
Cheers,
Jörg
On Thursday, November 8, 2012 5:02:31 PM UTC+1, Ivan Brusic wrote:
I am in the midst of tuning my Elasticsearch cluster. One major issue is
memory and some of the main culprits are facets over float values. To be
more precise, lots of distinct range facets over fields that contain prices
(11.99, 2139.00, 0.01, etc.). These ranges are combined on the client side
and mimic dynamic ranges.
Both ints and floats are 4 bytes. One common practice is to never use
floating points for currency due to rounding errors, but there are no
arithmetic operations performed on these values. Plus, I don't want to
convert legacy code. Is there any performance to be had from switching
values from floats to ints and performing divide by 100.0 on the client
side? I guess this question mainly pertains to Lucene since it deals with
the internal caches. The results can also be sorted on these values.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
Is there any performance to be had from switching