# Larger-than-memory mappings **URL:** \ **Category:** Elasticsearch\ **Created:** [January 23, 2013, 9:25pm UTC](https://discuss.elastic.co/t/larger-than-memory-mappings/10481 "2013-01-23T21:25:10Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![MarcusLongmuir](https://avatars.discourse-cdn.com/v4/letter/m/13edae/32.png) [@MarcusLongmuir](https://discuss.elastic.co/u/MarcusLongmuir)\ **Post date:** [January 23, 2013, 9:25pm UTC](https://discuss.elastic.co/t/larger-than-memory-mappings/10481/1 "2013-01-23T21:25:10Z")
I'm currently working on a project that will eventually have mappings larger than a single server's available memory. User's have the indirectly ability to create mappings so having the total size of the mappings limited to a single server's available memory is inhibitive. I've considered sharding per user (or groups of users), but this creates some constraints that I'd prefer not to have (searching across shards isn't preferable). Am I worrying over nothing? Is the total size of the mapping bound by single instance's available memory or disk space? -- ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [January 24, 2013, 5:00am UTC](https://discuss.elastic.co/t/larger-than-memory-mappings/10481/2 "2013-01-24T05:00:36Z")
It sounds like you are indeed worrying a bit too early. I assume you don't really mean that the mappings will be too big, but that the index will be large? ## Otis ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) On Wednesday, January 23, 2013 4:25:10 PM UTC-5, marcuslongmuir wrote: > I'm currently working on a project that will eventually have mappings > larger than a single server's available memory. User's have > the indirectly ability to create mappings so having the total size of the > mappings limited to a single server's available memory is inhibitive. > > I've considered sharding per user (or groups of users), but this creates > some constraints that I'd prefer not to have (searching across shards isn't > preferable). > > Am I worrying over nothing? Is the total size of the mapping bound by > single instance's available memory or disk space? -- ---
**Author:** ![MarcusLongmuir](https://avatars.discourse-cdn.com/v4/letter/m/13edae/32.png) [@MarcusLongmuir](https://discuss.elastic.co/u/MarcusLongmuir)\ **Post date:** [January 24, 2013, 5:04am UTC](https://discuss.elastic.co/t/larger-than-memory-mappings/10481/3 "2013-01-24T05:04:28Z")
I do mean mappings. I know that the indexed data can exceed the size of a single server, but I presume the assumption was made that mappings will never grow larger than the size of single server's available memory. I know that this isn't a common issue; it could only possibly occur if users were indirectly creating mappings, which in my case they are. On Thursday, January 24, 2013 5:00:36 AM UTC, Otis Gospodnetic wrote: > It sounds like you are indeed worrying a bit too early. > I assume you don't really mean that the mappings will be too big, but that > the index will be large? > > ## Otis > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > > On Wednesday, January 23, 2013 4:25:10 PM UTC-5, marcuslongmuir wrote: > > > I'm currently working on a project that will eventually have mappings > > larger than a single server's available memory. User's have > > the indirectly ability to create mappings so having the total size of the > > mappings limited to a single server's available memory is inhibitive. > > > > I've considered sharding per user (or groups of users), but this creates > > some constraints that I'd prefer not to have (searching across shards isn't > > preferable). > > > > Am I worrying over nothing? Is the total size of the mapping bound by > > single instance's available memory or disk space? -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [February 1, 2013, 11:23pm UTC](https://discuss.elastic.co/t/larger-than-memory-mappings/10481/4 "2013-02-01T23:23:26Z")
It will be an issue long before size of the mappings will reach the size of a single server's available memory (unless you are running elasticsearch on really tiny nodes). Mappings are part of cluster state and cluster state is maintained by master and published to every non-master node on every change. On Thursday, January 24, 2013 12:04:28 AM UTC-5, marcuslongmuir wrote: > I do mean mappings. I know that the indexed data can exceed the size of a > single server, but I presume the assumption was made that mappings will > never grow larger than the size of single server's available memory. > > I know that this isn't a common issue; it could only possibly occur if > users were indirectly creating mappings, which in my case they are. > > On Thursday, January 24, 2013 5:00:36 AM UTC, Otis Gospodnetic wrote: > > > It sounds like you are indeed worrying a bit too early. > > I assume you don't really mean that the mappings will be too big, but > > that the index will be large? > > > > ## Otis > > > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > > > > On Wednesday, January 23, 2013 4:25:10 PM UTC-5, marcuslongmuir wrote: > > > > > I'm currently working on a project that will eventually have mappings > > > larger than a single server's available memory. User's have > > > the indirectly ability to create mappings so having the total size of the > > > mappings limited to a single server's available memory is inhibitive. > > > > > > I've considered sharding per user (or groups of users), but this creates > > > some constraints that I'd prefer not to have (searching across shards isn't > > > preferable). > > > > > > Am I worrying over nothing? Is the total size of the mapping bound by > > > single instance's available memory or disk space? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:53am UTC](https://discuss.elastic.co/t/larger-than-memory-mappings/10481/5 "2017-07-06T02:53:26Z")
Closed on July 6, 2017, 2:53am UTC