# Standard query benchmark? **URL:** \ **Category:** Elasticsearch\ **Created:** [March 22, 2013, 5:22am UTC](https://discuss.elastic.co/t/standard-query-benchmark/11255 "2013-03-22T05:22:18Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [March 22, 2013, 5:22am UTC](https://discuss.elastic.co/t/standard-query-benchmark/11255/1 "2013-03-22T05:22:18Z")
Hi, Are there any standard-ish search benchmarking mechanisms in ES? I'm after running something that would allow me to compare "baseline queries" vs. the same queries with bloom filter turned on vs. the same queries with pulsing codec. Unless there is something standard-ish I could use, I'm tempted to just do this: - Index Wikipedia: [https://github.com/elasticsearch/stream2es](https://github.com/elasticsearch/stream2es) (with bloom, with pulsing, and baseline) - Use ActionGeneator to run queries: [https://github.com/sematext/ActionGenerator](https://github.com/sematext/ActionGenerator) against the index(es) - Use SPM to monitor and compare various metrics during/after the test If there are better ways to compare baseline vs. bloom filter vs, pulsing codec, I'm all eyeballs! ## Thanks, Otis ELASTICSEARCH Performance Monitoring - [http://sematext.com/spm/index.html](http://sematext.com/spm/index.html) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [March 22, 2013, 6:39pm UTC](https://discuss.elastic.co/t/standard-query-benchmark/11255/2 "2013-03-22T18:39:37Z")
there are no benchmark tools yet. I think you are stuck with what you lined up below. I am not sure if benchmarking is really needed at this point since unless you do a lot of key / value lookups those codecs won't buy you much and default codec pulses by default for very low docfreq terms. simon On Friday, March 22, 2013 6:22:18 AM UTC+1, Otis Gospodnetic wrote: > Hi, > > Are there any standard-ish search benchmarking mechanisms in ES? > I'm after running something that would allow me to compare "baseline > queries" vs. the same queries with bloom filter turned on vs. the same > queries with pulsing codec. > Unless there is something standard-ish I could use, I'm tempted to just do > this: > > - Index Wikipedia: [GitHub - elastic/stream2es: Stream data into ES (Wikipedia, Twitter, stdin, or other ESes)](https://github.com/elasticsearch/stream2es) (with > bloom, with pulsing, and baseline) > - Use ActionGeneator to run queries: > [https://github.com/sematext/ActionGenerator](https://github.com/sematext/ActionGenerator) against the index(es) > - Use SPM to monitor and compare various metrics during/after the test > > If there are better ways to compare baseline vs. bloom filter vs, pulsing > codec, I'm all eyeballs! > > ## Thanks, Otis > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [March 23, 2013, 3:43am UTC](https://discuss.elastic.co/t/standard-query-benchmark/11255/3 "2013-03-23T03:43:30Z")
Hi Simon, Aha, thanks for the info! If default already pulses just like what an explicit pulsing codec would do then maybe we can ignore that..... but still want to test bloom filter vs. the default. Indeed, lots of key lookups, that's what the queries we're looking at now are like. ## Otis ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) On Friday, March 22, 2013 2:39:37 PM UTC-4, simonw wrote: > there are no benchmark tools yet. I think you are stuck with what you > lined up below. > I am not sure if benchmarking is really needed at this point since unless > you do a lot of key / value lookups those codecs won't buy you much and > default codec pulses by default for very low docfreq terms. > > simon > > On Friday, March 22, 2013 6:22:18 AM UTC+1, Otis Gospodnetic wrote: > > > Hi, > > > > Are there any standard-ish search benchmarking mechanisms in ES? > > I'm after running something that would allow me to compare "baseline > > queries" vs. the same queries with bloom filter turned on vs. the same > > queries with pulsing codec. > > Unless there is something standard-ish I could use, I'm tempted to just > > do this: > > > > - Index Wikipedia: [GitHub - elastic/stream2es: Stream data into ES (Wikipedia, Twitter, stdin, or other ESes)](https://github.com/elasticsearch/stream2es) (with > > bloom, with pulsing, and baseline) > > - Use ActionGeneator to run queries: > > [https://github.com/sematext/ActionGenerator](https://github.com/sematext/ActionGenerator) against the index(es) > > - Use SPM to monitor and compare various metrics during/after the test > > > > If there are better ways to compare baseline vs. bloom filter vs, pulsing > > codec, I'm all eyeballs! > > > > ## Thanks, Otis > > > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [March 23, 2013, 12:15pm UTC](https://discuss.elastic.co/t/standard-query-benchmark/11255/4 "2013-03-23T12:15:15Z")
On Saturday, March 23, 2013 4:43:30 AM UTC+1, Otis Gospodnetic wrote: > Hi Simon, > > Aha, thanks for the info! If default already pulses just like what an > explicit pulsing codec would do then maybe we can ignore that..... but > still want to test bloom filter vs. the default. Indeed, lots of key > lookups, that's what the queries we're looking at now are like. FYI - Elasticsearch already uses bloom filter for the uid field. Using a bloom codec is only worth it when the likelyhood of a miss (the term is likely not in the term dict or only in one segment, just like uids) is very high. You are saving like a couple of micro-seconds and only if you really do this a lot like applying deletes in the IndexWriter this is worth the trouble. I totally wanna encourage you to try this and share your results, just wanna provide you with the info ahead of time. simon > ## Otis > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > > On Friday, March 22, 2013 2:39:37 PM UTC-4, simonw wrote: > > > there are no benchmark tools yet. I think you are stuck with what you > > lined up below. > > I am not sure if benchmarking is really needed at this point since unless > > you do a lot of key / value lookups those codecs won't buy you much and > > default codec pulses by default for very low docfreq terms. > > > > simon > > > > On Friday, March 22, 2013 6:22:18 AM UTC+1, Otis Gospodnetic wrote: > > > > > Hi, > > > > > > Are there any standard-ish search benchmarking mechanisms in ES? > > > I'm after running something that would allow me to compare "baseline > > > queries" vs. the same queries with bloom filter turned on vs. the same > > > queries with pulsing codec. > > > Unless there is something standard-ish I could use, I'm tempted to just > > > do this: > > > > > > - Index Wikipedia: [GitHub - elastic/stream2es: Stream data into ES (Wikipedia, Twitter, stdin, or other ESes)](https://github.com/elasticsearch/stream2es) (with > > > bloom, with pulsing, and baseline) > > > - Use ActionGeneator to run queries: > > > [https://github.com/sematext/ActionGenerator](https://github.com/sematext/ActionGenerator) against the index(es) > > > - Use SPM to monitor and compare various metrics during/after the > > > test > > > > > > If there are better ways to compare baseline vs. bloom filter vs, > > > pulsing codec, I'm all eyeballs! > > > > > > ## Thanks, Otis > > > > > > ELASTICSEARCH Performance Monitoring - > > > [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:45am UTC](https://discuss.elastic.co/t/standard-query-benchmark/11255/5 "2017-07-06T02:45:01Z")