# ES performance questions **URL:** \ **Category:** Elasticsearch\ **Created:** [December 13, 2011, 1:55am UTC](https://discuss.elastic.co/t/es-performance-questions/6134 "2011-12-13T01:55:02Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Chris\_Scribner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_scribner/32/3039_2.png) [@Chris\_Scribner](https://discuss.elastic.co/u/Chris_Scribner)\ **Post date:** [December 13, 2011, 1:55am UTC](https://discuss.elastic.co/t/es-performance-questions/6134/1 "2011-12-13T01:55:02Z")
Hi, Some co-workers and myself have been testing out ES recently. I wanted to ask about some observations we've made. Test server: 2x core i5, 4 GB RAM, 5400 RPM hard disk, Mac OS X Lion We attempted to insert 2 million documents with a name, description, and score. The index size was about 3 gigs. Doing a search with a query on description for two common words (common words are in about 10% of documents) takes about 6-7 seconds to return 500 results, ordered by score. It takes about 2 seconds to return the top 50 results. Given the specs of the computer running the search, this doesn't seem terrible. But, when running a search, we notice that ElasticSearch is using just a few % CPU time, and less than 150 MB of RAM, even though much more is available. That behavior makes me think the query latency is mainly time to read from the hard disk. But, I'm curious why ES isn't trying to use more RAM to make the query faster. I'm wondering if this all sounds normal, and whether there's anything we can do to optimize this particular type of search. We changed the index mapping to store the name, description, and score. In this case, we don't care about the total number of matches found, if that makes a difference. Thanks, Chris ---
**Author:** ![Patrick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/patrick/32/1437_2.png) [@Patrick](https://discuss.elastic.co/u/Patrick)\ **Post date:** [December 13, 2011, 2:34am UTC](https://discuss.elastic.co/t/es-performance-questions/6134/2 "2011-12-13T02:34:27Z")
Hey Chris, What do your java command line options look like? Also, did you check to see how much I/O it was doing at the time? Did you vary the search, or were you searching for similar/the same terms? ## Patrick > **[Patrick Ancillotti on about.me](https://about.me/patrick.ancillotti)** > > I live in New York. patrick eefy net On Mon, Dec 12, 2011 at 8:55 PM, Chris Scribner [scriby@gmail.com](mailto:scriby@gmail.com) wrote: > Hi, > > Some co-workers and myself have been testing out ES recently. I wanted > to ask about some observations we've made. > > Test server: 2x core i5, 4 GB RAM, 5400 RPM hard disk, Mac OS X Lion > > We attempted to insert 2 million documents with a name, description, > and score. The index size was about 3 gigs. > > Doing a search with a query on description for two common words > (common words are in about 10% of documents) takes about 6-7 seconds > to return 500 results, ordered by score. It takes about 2 seconds to > return the top 50 results. > > Given the specs of the computer running the search, this doesn't seem > terrible. But, when running a search, we notice that Elasticsearch is > using just a few % CPU time, and less than 150 MB of RAM, even though > much more is available. > > That behavior makes me think the query latency is mainly time to read > from the hard disk. But, I'm curious why ES isn't trying to use more > RAM to make the query faster. > > I'm wondering if this all sounds normal, and whether there's anything > we can do to optimize this particular type of search. We changed the > index mapping to store the name, description, and score. In this case, > we don't care about the total number of matches found, if that makes a > difference. > > Thanks, > > Chris ---
**Author:** ![Chris\_Scribner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_scribner/32/3039_2.png) [@Chris\_Scribner](https://discuss.elastic.co/u/Chris_Scribner)\ **Post date:** [December 13, 2011, 2:53am UTC](https://discuss.elastic.co/t/es-performance-questions/6134/3 "2011-12-13T02:53:22Z")
We tried passing -Xmx2g. We didn't check I/O stats on the process. We varied the search terms each query. Searching for the same search terms is very quick as expected (but not a particularly important use case for us). Chris On Dec 12, 9:34 pm, Patrick [patr...@eefy.net](mailto:patr...@eefy.net) wrote: > Hey Chris, > > What do your java command line options look like? Also, did you check to > see how much I/O it was doing at the time? Did you vary the search, or were > you searching for similar/the same terms? > > Patrick > ----------------------------------------[Patrick Ancillotti - New York | about.me](http://about.me/patrick.ancillotti) > patrick eefy net > > On Mon, Dec 12, 2011 at 8:55 PM, Chris Scribner [scr...@gmail.com](mailto:scr...@gmail.com) wrote: > > > Hi, > > > Some co-workers and myself have been testing out ES recently. I wanted > > to ask about some observations we've made. > > > Test server: 2x core i5, 4 GB RAM, 5400 RPM hard disk, Mac OS X Lion > > > We attempted to insert 2 million documents with a name, description, > > and score. The index size was about 3 gigs. > > > Doing a search with a query on description for two common words > > (common words are in about 10% of documents) takes about 6-7 seconds > > to return 500 results, ordered by score. It takes about 2 seconds to > > return the top 50 results. > > > Given the specs of the computer running the search, this doesn't seem > > terrible. But, when running a search, we notice that Elasticsearch is > > using just a few % CPU time, and less than 150 MB of RAM, even though > > much more is available. > > > That behavior makes me think the query latency is mainly time to read > > from the hard disk. But, I'm curious why ES isn't trying to use more > > RAM to make the query faster. > > > I'm wondering if this all sounds normal, and whether there's anything > > we can do to optimize this particular type of search. We changed the > > index mapping to store the name, description, and score. In this case, > > we don't care about the total number of matches found, if that makes a > > difference. > > > Thanks, > > > Chris ---
**Author:** ![Paul\_Loy](https://avatars.discourse-cdn.com/v4/letter/p/ad7895/32.png) [@Paul\_Loy](https://discuss.elastic.co/u/Paul_Loy)\ **Post date:** [December 13, 2011, 10:31am UTC](https://discuss.elastic.co/t/es-performance-questions/6134/4 "2011-12-13T10:31:12Z")
Did you let caches warm up before doing timings? As with all perf tests, you should always let both the JVM (hotspot) warm up as well as all the ES good stuff. Something else you can do would be to add more boxes to distribute the queries. If disk I/O is your bottleneck, sharing this load will speed things up. Especially if your data is ~3gb but your memory is only 2gb, maybe not all your data is fitting in memory. If every query is requiring disk access then you need more RAM - either on that one machine or by adding more machines. Paul. On Mon, Dec 12, 2011 at 6:53 PM, Chris Scribner [scriby@gmail.com](mailto:scriby@gmail.com) wrote: > We tried passing -Xmx2g. We didn't check I/O stats on the process. > > We varied the search terms each query. Searching for the same search > terms is very quick as expected (but not a particularly important use > case for us). > > Chris > > On Dec 12, 9:34 pm, Patrick [patr...@eefy.net](mailto:patr...@eefy.net) wrote: > > > Hey Chris, > > > > What do your java command line options look like? Also, did you check to > > see how much I/O it was doing at the time? Did you vary the search, or > > were > > you searching for similar/the same terms? > > > > ## Patrick > > [Patrick Ancillotti - New York | about.me](http://about.me/patrick.ancillotti) > > > patrick eefy net > > > > On Mon, Dec 12, 2011 at 8:55 PM, Chris Scribner [scr...@gmail.com](mailto:scr...@gmail.com) > > wrote: > > > > > Hi, > > > > > Some co-workers and myself have been testing out ES recently. I wanted > > > to ask about some observations we've made. > > > > > Test server: 2x core i5, 4 GB RAM, 5400 RPM hard disk, Mac OS X Lion > > > > > We attempted to insert 2 million documents with a name, description, > > > and score. The index size was about 3 gigs. > > > > > Doing a search with a query on description for two common words > > > (common words are in about 10% of documents) takes about 6-7 seconds > > > to return 500 results, ordered by score. It takes about 2 seconds to > > > return the top 50 results. > > > > > Given the specs of the computer running the search, this doesn't seem > > > terrible. But, when running a search, we notice that Elasticsearch is > > > using just a few % CPU time, and less than 150 MB of RAM, even though > > > much more is available. > > > > > That behavior makes me think the query latency is mainly time to read > > > from the hard disk. But, I'm curious why ES isn't trying to use more > > > RAM to make the query faster. > > > > > I'm wondering if this all sounds normal, and whether there's anything > > > we can do to optimize this particular type of search. We changed the > > > index mapping to store the name, description, and score. In this case, > > > we don't care about the total number of matches found, if that makes a > > > difference. > > > > > Thanks, > > > > > Chris ## -- Paul Loy [paul@keteracel.com](mailto:paul@keteracel.com) [http://uk.linkedin.com/in/paulloy](http://uk.linkedin.com/in/paulloy) ---
**Author:** ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)\ **Post date:** [December 13, 2011, 10:58am UTC](https://discuss.elastic.co/t/es-performance-questions/6134/5 "2011-12-13T10:58:05Z")
> The index size was about 3 gigs. Hmmh, that should not be too much for the server... Some questions 😉 Did you verified that ES is really using that 2gig? Are you starting ES via the elasticsearch scipt, then there would be a variable for this. How many shards do you have? Have you changed any other settings or using different index settings? Is it a simple term query? Which java version are you using and did you use "-server"? Peter. ---
**Author:** ![Chris\_Scribner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_scribner/32/3039_2.png) [@Chris\_Scribner](https://discuss.elastic.co/u/Chris_Scribner)\ **Post date:** [December 13, 2011, 7:48pm UTC](https://discuss.elastic.co/t/es-performance-questions/6134/6 "2011-12-13T19:48:00Z")
We are starting elasticsearch via the shell script, and setting memory variables (ES\_MIN\_MEM, ES\_MAX\_MEM). Shards: Whatever the default is (1, I think) Index: We haven't changed any other settings. We ran the tests again today with the automatically created index, and saw similar performance. Query: { fields: ["goodness"], sort: [{goodness: {order: "desc"}}], query: { text: { description: { query: "randomWord1 randomWord2", operator: "and" } } }, size: 50 } Java version: 1.6.0\_29. We did not use "-server" As far as we can tell, ES is not using all the memory allocated. On the same machine we tested on yesterday, it gets up to a max of about 200-250 MB. Watching the disk I/O, it's only reading about 700 KB/s from the HD while the queries are running. We tested on a different machine running Ubuntu, and it got up to about 350 MB. We verified that it was respecting the memory settings, because when it builds the index it used up all the RAM allocated. We built a test script that runs searches repeatedly (100 - 1000 at a time). Running 100 searches (in parallel) (using the query above) takes between 7-20 seconds, depending on how common the words are we search with. (As expected, the search runs more slowly at first before the cache is primed. The previous numbers are where it levels out to after a few runs) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 16, 2011, 3:26pm UTC](https://discuss.elastic.co/t/es-performance-questions/6134/7 "2011-12-16T15:26:15Z")
Regarding the amount of memory, elasticsearch (and Lucene) will use the memory it needs, not more if there is more available. At least when search is executed. Hard to tell with the machine you have if its really slow or not, if you want to zip your data directory and dropbox it, I can run the query and check here. On Tue, Dec 13, 2011 at 9:48 PM, Chris Scribner [scriby@gmail.com](mailto:scriby@gmail.com) wrote: > We are starting elasticsearch via the shell script, and setting memory > variables (ES\_MIN\_MEM, ES\_MAX\_MEM). > Shards: Whatever the default is (1, I think) > Index: We haven't changed any other settings. We ran the tests again > today with the automatically created index, and saw similar > performance. > Query: > { > fields: ["goodness"], > sort: [{goodness: {order: "desc"}}], > query: { > text: { > description: { > query: "randomWord1 randomWord2", > operator: "and" > } > } > }, > size: 50 > } > > Java version: 1.6.0\_29. We did not use "-server" > > As far as we can tell, ES is not using all the memory allocated. On > the same machine we tested on yesterday, it gets up to a max of about > 200-250 MB. Watching the disk I/O, it's only reading about 700 KB/s > from the HD while the queries are running. We tested on a different > machine running Ubuntu, and it got up to about 350 MB. We verified > that it was respecting the memory settings, because when it builds the > index it used up all the RAM allocated. > > We built a test script that runs searches repeatedly (100 - 1000 at a > time). Running 100 searches (in parallel) (using the query above) > takes between 7-20 seconds, depending on how common the words are we > search with. (As expected, the search runs more slowly at first before > the cache is primed. The previous numbers are where it levels out to > after a few runs) ---
**Author:** ![Chris\_Scribner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_scribner/32/3039_2.png) [@Chris\_Scribner](https://discuss.elastic.co/u/Chris_Scribner)\ **Post date:** [December 16, 2011, 5:01pm UTC](https://discuss.elastic.co/t/es-performance-questions/6134/8 "2011-12-16T17:01:52Z")
Shay, Thanks for the response. We've since done some testing on "production grade" boxes and the performance is quite acceptable. The guess we made from this data is that Lucene doesn't attempt to cache the underlying documents in memory -- it just stores the index in memory. Does that sound about right? Thanks, Chris ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 16, 2011, 5:27pm UTC](https://discuss.elastic.co/t/es-performance-questions/6134/9 "2011-12-16T17:27:14Z")
It loads part of the terms of the inverted index in memory. You can control that part using the term index interval and divisor: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/). On Fri, Dec 16, 2011 at 7:01 PM, Chris Scribner [scriby@gmail.com](mailto:scriby@gmail.com) wrote: > Shay, > > Thanks for the response. We've since done some testing on "production > grade" boxes and the performance is quite acceptable. > > The guess we made from this data is that Lucene doesn't attempt to > cache the underlying documents in memory -- it just stores the index > in memory. Does that sound about right? > > Thanks, > > Chris ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:45am UTC](https://discuss.elastic.co/t/es-performance-questions/6134/10 "2017-07-06T03:45:09Z")