# Index performace with large arrays **URL:** \ **Category:** Elasticsearch\ **Created:** [July 21, 2014, 2:58pm UTC](https://discuss.elastic.co/t/index-performace-with-large-arrays/18765 "2014-07-21T14:58:16Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![SteveM](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@SteveM](https://discuss.elastic.co/u/SteveM)\ **Post date:** [July 21, 2014, 2:58pm UTC](https://discuss.elastic.co/t/index-performace-with-large-arrays/18765/1 "2014-07-21T14:58:16Z")
Hi there. I'm new to ES and would appreciate some advice on design concepts around large arrays. I am writing a help tip feature that pops up a message each time a user logs in. The user can flag a checkbox if they do not want to see this particular tip again. After playing with ElasticSearch the solution I came up with involved using a HelpTip document which contains an array of UserIds (identifying the users who have flagged that they do not want to see this tip again). Example1: HelpTip { "title": "Need help getting started?", "text": "Watch our overview video", "userArray": ["id1", "id2"] } I know ES can cope with large arrays but I wonder if there would be performance issues if this array grew to 4000+ IDs. This record would be regularly re-indexed (each time a new user ID is added to the array). would there be performance issues when indexing a document containing a large array field? Is this a sensible approach or would I be better using a relational model and holding the Help Tip info and the list of users in separate documents, then parsing them using two separate calls from my application? Example 2: HelpTip { "title": "Need help getting started?", "text": "Watch our overview video" } HelpTipUserFlags { HelpTipId: "1", UserId: "ID1" } Hope this makes sense. Thanks in advance for any help. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9cba2c32-6266-4b87-b708-83ee64499dbf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9cba2c32-6266-4b87-b708-83ee64499dbf%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 21, 2014, 3:56pm UTC](https://discuss.elastic.co/t/index-performace-with-large-arrays/18765/2 "2014-07-21T15:56:13Z")
If the user can opt out, I assume you have fewer opt outs than opt ins, then you should use opt outs for an andnot filter 🙂 In that case, I would create an opt out index in the form index/type/id users/optouts/ with docs containing a quite short array of opt outs { "optouts: ["id1", "id2", ... "idn"] } so you can get the doc, read the opt out array, and add it as an "and not" filter to your help tip query. You could also add this optouts array to the user index, but this depends on your overall design. If you want to remove the opt outs, you could simply drop the optputs mapping type. Regarding the array length, you can add as much values as you like, ES can handle that. If the docs get long (I mean thousands of entries), they will take substantial time just for fetching them, so I think you should prefer a model with data as short as possible. Jörg On Mon, Jul 21, 2014 at 4:58 PM, Steve Mee [steve@genialgenetics.com](mailto:steve@genialgenetics.com) wrote: > Hi there. I'm new to ES and would appreciate some advice on design > concepts around large arrays. > > I am writing a help tip feature that pops up a message each time a user > logs in. The user can flag a checkbox if they do not want to see this > particular tip again. > > After playing with Elasticsearch the solution I came up with involved > using a HelpTip document which contains an array of UserIds (identifying > the users who have flagged that they do not want to see this tip again). > > Example1: > HelpTip > { > "title": "Need help getting started?", > "text": "Watch our overview video", > "userArray": ["id1", "id2"] > } > > I know ES can cope with large arrays but I wonder if there would be > performance issues if this array grew to 4000+ IDs. This record would be > regularly re-indexed (each time a new user ID is added to the array). would > there be performance issues when indexing a document containing a large > array field? > > Is this a sensible approach or would I be better using a relational model > and holding the Help Tip info and the list of users in separate documents, > then parsing them using two separate calls from my application? > Example 2: > HelpTip > { > "title": "Need help getting started?", > "text": "Watch our overview video" > } > > HelpTipUserFlags > { > HelpTipId: "1", > UserId: "ID1" > } > > Hope this makes sense. Thanks in advance for any help. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/9cba2c32-6266-4b87-b708-83ee64499dbf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9cba2c32-6266-4b87-b708-83ee64499dbf%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/9cba2c32-6266-4b87-b708-83ee64499dbf%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/9cba2c32-6266-4b87-b708-83ee64499dbf%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFw7P8FQu9%2Bc2ajj0Vg2wNvbpz%3D%2Bo9Af8R-5p9Cj8-7FQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFw7P8FQu9%2Bc2ajj0Vg2wNvbpz%3D%2Bo9Af8R-5p9Cj8-7FQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![SteveM](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@SteveM](https://discuss.elastic.co/u/SteveM)\ **Post date:** [July 21, 2014, 4:10pm UTC](https://discuss.elastic.co/t/index-performace-with-large-arrays/18765/3 "2014-07-21T16:10:13Z")
> Thanks for the response Jörg. That tells me exactly what I need to know... > stay away from very large arrays here in my design 🙂 Cheers - Steve -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f06b544f-8aec-4c44-aa38-ce53e5f0be74%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f06b544f-8aec-4c44-aa38-ce53e5f0be74%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:14am UTC](https://discuss.elastic.co/t/index-performace-with-large-arrays/18765/4 "2017-07-06T01:14:31Z")