# Can we use elasticsearch do analysis work? **URL:** \ **Category:** Elasticsearch\ **Created:** [March 7, 2016, 2:43am UTC](https://discuss.elastic.co/t/can-we-use-elasticsearch-do-analysis-work/43611 "2016-03-07T02:43:07Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Kramer\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kramer_li/32/46872_2.png) [@Kramer\_Li](https://discuss.elastic.co/u/Kramer_Li)\ **Post date:** [March 7, 2016, 2:43am UTC](https://discuss.elastic.co/t/can-we-use-elasticsearch-do-analysis-work/43611/1 "2016-03-07T02:43:07Z")
Hi All, Thanks for your interest. I know that elasticsearch announce that some company build cluster including hundreds of computers to do analysis works. I`m not sure what kind of analysis jobs those company is doing. But I have some requirement like this. I have some data like this. A structured data. Very simple. ```auto gender, age, school, name, score male, 18, s1, kramer, 100 male, 15, s2, dav, 88 ``` I want to run the analysis like ```auto SELECT school, avg(score) FROM table GROUP BY school ORDER BY avg(score) ``` I know ES can do this. But I`m worrying its performance （Because the table may contain **more than ten billions** docs, also the actual data structure will be more complicated than here）. So I did a test on a computer like : ``` CPU 8core Memory 16G OS centOS 7 elasticsearch 2.1 8G memory for ES heap no swap ``` The way I do my test is : **First , find out how many docs one shard can process. In my computer, it is 400 000** **Second, find out how many shards my computer can hold. In my computer, it is 2.** The detail of my test is here > [@Is this a right way to do performance evaluation?](https://discuss.elastic.co/t/is-this-a-right-way-to-do-performance-evaluation/43476/1): > > Hi I am doing a performance evaluation with 2 queries. The purpose is to figure out how many docs one node can process. But Im not sure if Im using the correct way to test. Also not sure if the conclusion I got is correct. Because they seems too small. Below are the hardware/software env I`m using. CPU 8core Memory 16G OS centOS 7 elasticsearch 2.1 8G memory for ES heap no swap The requirements is that all query response time less 3 seconds. The map I use is like below: "@timestamp": { "… So totally, one computer can analysis 800 000 docs. If I want to analysis 10 billion docs, I need 12500 computers. It is ridiculous... **So I am wondering does ES can really do the analysis above?** **Also I noticed there is a product called Elasticsearch-Hadoop. I haven`t read the corresponding docs yet. But it looks like something used for analysis works. So am I using the wrong products?** ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 7, 2016, 3:00am UTC](https://discuss.elastic.co/t/can-we-use-elasticsearch-do-analysis-work/43611/2 "2016-03-07T03:00:17Z")
Yes, ES can do this sort of thing. > [@Kramer\_Li](#): > > one computer can analysis 800 000 docs. If I want to analysis 10 billion docs, I need 12500 computers. It is ridiculous You are talking about a laptop here, so your extrapolation is not accurate. Try running this on an actual dedicated server instead. ---
**Author:** ![Kramer\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kramer_li/32/46872_2.png) [@Kramer\_Li](https://discuss.elastic.co/u/Kramer_Li)\ **Post date:** [March 7, 2016, 3:28am UTC](https://discuss.elastic.co/t/can-we-use-elasticsearch-do-analysis-work/43611/3 "2016-03-07T03:28:37Z")
Hi Thanks for your reply. So normally what kind of computer people use when they setup ES cluster for analysis? Like the **physical memory , CPU core numbers etc**. I do not have much experience on this. But obviously use ten thousands of computer to do analysis on 10 billion data is not a good choice Regards Mingwei ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 7, 2016, 8:52pm UTC](https://discuss.elastic.co/t/can-we-use-elasticsearch-do-analysis-work/43611/4 "2016-03-07T20:52:32Z")
I can't answer that because there is no normal. But start with something _dedicated_, at least 4 cores, 8GB of RAM and some disk. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:10pm UTC](https://discuss.elastic.co/t/can-we-use-elasticsearch-do-analysis-work/43611/5 "2017-07-05T23:10:36Z")