# Slow search perfomance when using mmap versus memory
**URL:** https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050
**Category:** Elasticsearch
**Created:** [June 11, 2014, 4:44pm UTC](https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050 "2014-06-11T16:44:52Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@MikeP\_2](https://discuss.elastic.co/u/MikeP_2)
#### Post date: [June 11, 2014, 4:44pm UTC](https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050/1 "2014-06-11T16:44:52Z")
Our servers have 130 GB of RAM and we are giving ElasticSearch 30GB of
heap. Each machine contains one shard that is about 3GB with about 1.3
million documents. When we search using a match query and the "memory"
store type, the queries are about twice as fast as when we use "mmap (with
mlockall enabled)". Are there any tools to to troubleshoot the latency?
Thanks
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [June 11, 2014, 9:09pm UTC](https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050/2 "2014-06-11T21:09:40Z")
Can you share your setup configuration, and an example document and a
query? So it is possible to recreate your situation?
Also interesting would be OS version, ES version, Java JVM version.
Thanks,
Jörg
On Wed, Jun 11, 2014 at 6:44 PM, MikeP [michaelp79@gmail.com](mailto:michaelp79@gmail.com) wrote:
> Our servers have 130 GB of RAM and we are giving Elasticsearch 30GB of
> heap. Each machine contains one shard that is about 3GB with about 1.3
> million documents. When we search using a match query and the "memory"
> store type, the queries are about twice as fast as when we use "mmap (with
> mlockall enabled)". Are there any tools to to troubleshoot the latency?
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com)
> [https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com?utm_medium=email&utm_source=footer)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFgyB3M1121BOSaKxefTT\_XK1sgZLLKxj-GSKZSedCVKQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFgyB3M1121BOSaKxefTT_XK1sgZLLKxj-GSKZSedCVKQ%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [June 11, 2014, 9:25pm UTC](https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050/3 "2014-06-11T21:25:07Z")
N.B. of course, also the response times you measured and how you measuring
them are of interest.
In my tests with ES 1.2.1 and JDK 8u5 on Mac OS X, with a test index of 1.5
mio docs, I see search times of around 1-5ms on mmapfs for random match
queries (except the first query after the node started). Index store
"memory" is not faster.
Jörg
On Wed, Jun 11, 2014 at 11:09 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \<
[joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote:
> Can you share your setup configuration, and an example document and a
> query? So it is possible to recreate your situation?
>
> Also interesting would be OS version, ES version, Java JVM version.
>
> Thanks,
>
> Jörg
>
> On Wed, Jun 11, 2014 at 6:44 PM, MikeP [michaelp79@gmail.com](mailto:michaelp79@gmail.com) wrote:
>
> > Our servers have 130 GB of RAM and we are giving Elasticsearch 30GB of
> > heap. Each machine contains one shard that is about 3GB with about 1.3
> > million documents. When we search using a match query and the "memory"
> > store type, the queries are about twice as fast as when we use "mmap (with
> > mlockall enabled)". Are there any tools to to troubleshoot the latency?
> >
> > Thanks
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHQhi1V0oWi7dvV\_1sj\_uDVBqFcG05uPS8RGaNdw%2Bgu5w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHQhi1V0oWi7dvV_1sj_uDVBqFcG05uPS8RGaNdw%2Bgu5w%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@MikeP\_2](https://discuss.elastic.co/u/MikeP_2)
#### Post date: [June 11, 2014, 11:19pm UTC](https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050/4 "2014-06-11T23:19:54Z")
Hi Jörg,
I'm using ES 1.1.1, JDK7u55, RHEL5. Our documents contains only integers
between 0-1,000,000. Our query is a bit unusual as it is basically a match
query that only contains ORs and there could be up to 1,000 clauses. For
example:
{"query": {
"match":{
"num\_field":{
"query":"1 2 3 4 5 99998 99999","operator":"or"}
}
}
}
When we search using less than 100 clauses, the search is pretty quick,
typically less 30ms on both "memory" and "mmap". When there are 1,000
clauses, the average is about 200ms using "memory" and about 400ms using
"mmap".
Thanks for the help.
On Wednesday, June 11, 2014 2:25:14 PM UTC-7, Jörg Prante wrote:
> N.B. of course, also the response times you measured and how you measuring
> them are of interest.
>
> In my tests with ES 1.2.1 and JDK 8u5 on Mac OS X, with a test index of
> 1.5 mio docs, I see search times of around 1-5ms on mmapfs for random match
> queries (except the first query after the node started). Index store
> "memory" is not faster.
>
> Jörg
>
> On Wed, Jun 11, 2014 at 11:09 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) \ \<
> [joerg...@gmail.com](mailto:joerg...@gmail.com) \\> wrote:
>
> > Can you share your setup configuration, and an example document and a
> > query? So it is possible to recreate your situation?
> >
> > Also interesting would be OS version, ES version, Java JVM version.
> >
> > Thanks,
> >
> > Jörg
> >
> > On Wed, Jun 11, 2014 at 6:44 PM, MikeP \<[micha...@gmail.com](mailto:micha...@gmail.com) \\>
> > wrote:
> >
> > > Our servers have 130 GB of RAM and we are giving Elasticsearch 30GB of
> > > heap. Each machine contains one shard that is about 3GB with about 1.3
> > > million documents. When we search using a match query and the "memory"
> > > store type, the queries are about twice as fast as when we use "mmap (with
> > > mlockall enabled)". Are there any tools to to troubleshoot the latency?
> > >
> > > Thanks
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > To view this discussion on the web visit
> > > [https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com)
> > > [https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > .
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1b324e89-6d91-453d-981a-94f411171e4f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b324e89-6d91-453d-981a-94f411171e4f%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [June 12, 2014, 10:58am UTC](https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050/5 "2014-06-12T10:58:30Z")
You should use a boolean query and wrap it into a constant core query.
Constant score query is important, otherwise each clause will lead to score
calculation which has a significant impact on the overall search response
time.
There is also a notable difference of performance on AWS between "memory"
and "mmapfs" because AWS VMs are slow - this is another story, not related
to queries.
Jörg
On Thu, Jun 12, 2014 at 1:19 AM, MikeP [michaelp79@gmail.com](mailto:michaelp79@gmail.com) wrote:
> Hi Jörg,
>
> I'm using ES 1.1.1, JDK7u55, RHEL5. Our documents contains only integers
> between 0-1,000,000. Our query is a bit unusual as it is basically a match
> query that only contains ORs and there could be up to 1,000 clauses. For
> example:
>
> {"query": {
> "match":{
> "num\_field":{
> "query":"1 2 3 4 5 99998 99999","operator":"or"}
> }
> }
> }
>
> When we search using less than 100 clauses, the search is pretty quick,
> typically less 30ms on both "memory" and "mmap". When there are 1,000
> clauses, the average is about 200ms using "memory" and about 400ms using
> "mmap".
>
> Thanks for the help.
>
> On Wednesday, June 11, 2014 2:25:14 PM UTC-7, Jörg Prante wrote:
>
> > N.B. of course, also the response times you measured and how you
> > measuring them are of interest.
> >
> > In my tests with ES 1.2.1 and JDK 8u5 on Mac OS X, with a test index of
> > 1.5 mio docs, I see search times of around 1-5ms on mmapfs for random match
> > queries (except the first query after the node started). Index store
> > "memory" is not faster.
> >
> > Jörg
> >
> > On Wed, Jun 11, 2014 at 11:09 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) [joerg...@gmail.com](mailto:joerg...@gmail.com)
> > wrote:
> >
> > > Can you share your setup configuration, and an example document and a
> > > query? So it is possible to recreate your situation?
> > >
> > > Also interesting would be OS version, ES version, Java JVM version.
> > >
> > > Thanks,
> > >
> > > Jörg
> > >
> > > On Wed, Jun 11, 2014 at 6:44 PM, MikeP [micha...@gmail.com](mailto:micha...@gmail.com) wrote:
> > >
> > > > Our servers have 130 GB of RAM and we are giving Elasticsearch 30GB of
> > > > heap. Each machine contains one shard that is about 3GB with about 1.3
> > > > million documents. When we search using a match query and the "memory"
> > > > store type, the queries are about twice as fast as when we use "mmap (with
> > > > mlockall enabled)". Are there any tools to to troubleshoot the latency?
> > > >
> > > > Thanks
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > >
> > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%
> > > > [40googlegroups.com](http://40googlegroups.com)
> > > > [https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/afde39cc-e0c4-49b8-b3ec-a107bf8a4755%40googlegroups.com?utm_medium=email&utm_source=footer)
> > > > .
> > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/1b324e89-6d91-453d-981a-94f411171e4f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1b324e89-6d91-453d-981a-94f411171e4f%40googlegroups.com)
> > [https://groups.google.com/d/msgid/elasticsearch/1b324e89-6d91-453d-981a-94f411171e4f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1b324e89-6d91-453d-981a-94f411171e4f%40googlegroups.com?utm_medium=email&utm_source=footer)
> > .
>
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFCwBrBQczNAaXE-LBQxd1HpO9hdDX6\_GY9X-0dt0tAzg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFCwBrBQczNAaXE-LBQxd1HpO9hdDX6_GY9X-0dt0tAzg%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@MikeP\_2](https://discuss.elastic.co/u/MikeP_2)
#### Post date: [June 12, 2014, 7:43pm UTC](https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050/6 "2014-06-12T19:43:06Z")
We actually need the score for each. When we run the same query directly on
Lucene (no Elasticsearch), the response times are fast, typically under
100ms. We're trying to understand why there is such a large difference in
response times when search using Elasticsearch versus Lucene.
On Thursday, June 12, 2014 3:58:35 AM UTC-7, Jörg Prante wrote:
> You should use a boolean query and wrap it into a constant core query.
> Constant score query is important, otherwise each clause will lead to score
> calculation which has a significant impact on the overall search response
> time.
>
> There is also a notable difference of performance on AWS between "memory"
> and "mmapfs" because AWS VMs are slow - this is another story, not related
> to queries.
>
> Jörg
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ce261303-00ac-40c3-88ad-4d296ebd9a07%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ce261303-00ac-40c3-88ad-4d296ebd9a07%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:22am UTC](https://discuss.elastic.co/t/slow-search-perfomance-when-using-mmap-versus-memory/18050/7 "2017-07-06T01:22:55Z")