# Huge database I'm wondering if Elasticsearch can handle **URL:** \ **Category:** Elasticsearch\ **Created:** [September 9, 2015, 10:53pm UTC](https://discuss.elastic.co/t/huge-database-im-wondering-if-elasticsearch-can-handle/28974 "2015-09-09T22:53:40Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![razvant](https://avatars.discourse-cdn.com/v4/letter/r/a87d85/32.png) [@razvant](https://discuss.elastic.co/u/razvant)\ **Post date:** [September 9, 2015, 10:53pm UTC](https://discuss.elastic.co/t/huge-database-im-wondering-if-elasticsearch-can-handle/28974/1 "2015-09-09T22:53:40Z")
Hello! I need to store a maximum of 200 billion records, each having 2 fields: A, and B, both strings of no more than 255 characters. I'll be inserting records into this database at a pace of about 50 000 per second. About once a second, I'll also be querying the database. All the queries will be the same: I'll need all the records where the field A=X, for a given X string. 1. Is it possible to use Elastic Search to store such a database? 2. What kind of hardware would I need to store it? Thanks, Razvan ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 10, 2015, 3:49am UTC](https://discuss.elastic.co/t/huge-database-im-wondering-if-elasticsearch-can-handle/28974/2 "2015-09-10T03:49:34Z")
Is this the only way you're going to use the database? Asking the whole dataset the same question every second isn't efficient since you only need to consider the data that has been inserted since the last time. I'd consider pushing the records to a broker and have a program process each new record and keep track of how many there are with A=X. If events expire one would have to deal with that in some way too. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 10, 2015, 4:34am UTC](https://discuss.elastic.co/t/huge-database-im-wondering-if-elasticsearch-can-handle/28974/3 "2015-09-10T04:34:20Z")
ES can handle this, using filtered queries will be the optimal way too. ---
**Author:** ![razvant](https://avatars.discourse-cdn.com/v4/letter/r/a87d85/32.png) [@razvant](https://discuss.elastic.co/u/razvant)\ **Post date:** [September 10, 2015, 4:34pm UTC](https://discuss.elastic.co/t/huge-database-im-wondering-if-elasticsearch-can-handle/28974/4 "2015-09-10T16:34:55Z")
I wouldn't be asking the exact same question. X is different on every query. ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [September 11, 2015, 12:56am UTC](https://discuss.elastic.co/t/huge-database-im-wondering-if-elasticsearch-can-handle/28974/5 "2015-09-11T00:56:38Z")
Yes, ES should be fine with this.... if you give it enough/appropriate hardware, of course. Super small/narrow docs and the cheapest possible queries. Just don't go updating a large number of docs at once and try to avoid shard reallocation. ## Otis Monitoring \* Alerting \* Anomaly Detection \* Centralized Log Management Elasticsearch Consulting & Support \* [http://sematext.com/](http://sematext.com/) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:51pm UTC](https://discuss.elastic.co/t/huge-database-im-wondering-if-elasticsearch-can-handle/28974/6 "2017-07-05T23:51:01Z")