# Several questions on ES in production environment **URL:** \ **Category:** Elasticsearch\ **Created:** [December 24, 2013, 1:47pm UTC](https://discuss.elastic.co/t/several-questions-on-es-in-production-environment/15011 "2013-12-24T13:47:54Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Han\_JU](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/han_ju/32/962_2.png) [@Han\_JU](https://discuss.elastic.co/u/Han_JU)\ **Post date:** [December 24, 2013, 1:47pm UTC](https://discuss.elastic.co/t/several-questions-on-es-in-production-environment/15011/1 "2013-12-24T13:47:54Z")
Hi, We're approaching the first release of our product and we use ElasticSearch as a key component in our system. But there's still some questions and doubts so I'd like to listen to the more experienced users and ElasticSearch folks here. 1. We use ElasticSearch as a search tool but also the storage of all documents. It means that the front-end retrieves fields from ES just as if it's a database. We've already disable the index (index: no) on the fields that don't need to be searched (list of ids etc.) but is this a good usage of ElasticSearch? Given that we expected to have ~ 1 billion documents (~ 1.4kb each) in our first 3 months in a single index. 2. We will use thrift to push documents in production because we've seen a performance gain. Is there any downside of using thrift over plain json? 3. Some of our queries uses regexp filter. In my comprehension this needs to load the target field of every document to see if it matches, so it's pretty costly for an index of 1 billion docs? We're also benchmarking our ES setup but your advice and experience are very appreciated! Thanks in advance! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/563d3c8f-cb3c-4c8c-a13d-2f8fcf6c42ce%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/563d3c8f-cb3c-4c8c-a13d-2f8fcf6c42ce%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [December 25, 2013, 3:44am UTC](https://discuss.elastic.co/t/several-questions-on-es-in-production-environment/15011/2 "2013-12-25T03:44:11Z")
Hi, On Tuesday, December 24, 2013 8:47:54 AM UTC-5, Han JU wrote: > Hi, > > We're approaching the first release of our product and we use > Elasticsearch as a key component in our system. But there's still some > questions and doubts so I'd like to listen to the more experienced users > and Elasticsearch folks here. > > 1. We use Elasticsearch as a search tool but also the storage of all > documents. It means that the front-end retrieves fields from ES just as if > it's a database. We've already disable the index (index: no) on the fields > that don't need to be searched (list of ids etc.) but is this a good usage > of Elasticsearch? Given that we expected to have ~ 1 billion documents (~ > 1.4kb each) in our first 3 months in a single index. 1.4KB is pretty small, so that's fine. Often keeping it all in ES is simpler - doesn't require another hope to another server (e.g. a DB) to retrieve display data, there is one moving piece fewer, which makes everything simple. I'd keep your display data in ES and worry about changing it later IFF you have issues. > 1. We will use thrift to push documents in production because we've seen a > performance gain. Is there any downside of using thrift over plain json? > > 2. Some of our queries uses regexp filter. In my comprehension this needs > to load the target field of every document to see if it matches, so it's > pretty costly for an index of 1 billion docs? Yes, regexps are not the fastest. What are you trying to do that requires regexp filter? ## Otis Performance Monitoring \* Log Analytics \* Search Analytics Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f26e7656-61cf-4609-8182-d7c6d406a5cc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f26e7656-61cf-4609-8182-d7c6d406a5cc%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Han\_JU](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/han_ju/32/962_2.png) [@Han\_JU](https://discuss.elastic.co/u/Han_JU)\ **Post date:** [December 26, 2013, 9:40am UTC](https://discuss.elastic.co/t/several-questions-on-es-in-production-environment/15011/3 "2013-12-26T09:40:12Z")
Thanks Otis. The reason that we use regexp filter is also the simplicity. We have a layer that translates front-end format query to Elasticsearch query DSL. Definitly we need use something like phrase\_query and some custom analyser. 在 2013年12月25日星期三UTC+1上午4时44分11秒，Otis Gospodnetic写道： > Hi, > > On Tuesday, December 24, 2013 8:47:54 AM UTC-5, Han JU wrote: > > > Hi, > > > > We're approaching the first release of our product and we use > > Elasticsearch as a key component in our system. But there's still some > > questions and doubts so I'd like to listen to the more experienced users > > and Elasticsearch folks here. > > > > 1. We use Elasticsearch as a search tool but also the storage of all > > documents. It means that the front-end retrieves fields from ES just as if > > it's a database. We've already disable the index (index: no) on the fields > > that don't need to be searched (list of ids etc.) but is this a good usage > > of Elasticsearch? Given that we expected to have ~ 1 billion documents (~ > > 1.4kb each) in our first 3 months in a single index. > > 1.4KB is pretty small, so that's fine. Often keeping it all in ES is > simpler - doesn't require another hope to another server (e.g. a DB) to > retrieve display data, there is one moving piece fewer, which makes > everything simple. I'd keep your display data in ES and worry about > changing it later IFF you have issues. > > > 1. We will use thrift to push documents in production because we've seen > > a performance gain. Is there any downside of using thrift over plain json? > > > > 2. Some of our queries uses regexp filter. In my comprehension this needs > > to load the target field of every document to see if it matches, so it's > > pretty costly for an index of 1 billion docs? > > Yes, regexps are not the fastest. What are you trying to do that requires > regexp filter? > > ## Otis > > Performance Monitoring \* Log Analytics \* Search Analytics > Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4665964d-b3f2-42ca-980f-b543496686f5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4665964d-b3f2-42ca-980f-b543496686f5%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:59am UTC](https://discuss.elastic.co/t/several-questions-on-es-in-production-environment/15011/4 "2017-07-06T01:59:23Z")
Closed on July 6, 2017, 1:59am UTC