How Elasticsearch Working With Big Data?


(Truong Van Linh) #1

Hi everyone,

Can you explain me in detail how Elasticsearch working with Big Data? Elasticsearch mechanism when processing Big Data?

I'm really really need information about Elasticsearch mechanism when its processing Big Data.

Best Regards


(Magnus Bäck) #2

This question is too open-ended for anyone to give you a good answer. Please try to be more specific.


(Truong Van Linh) #3

Hi @magnusbaeck,
I mean how Elasticsearch indexing, Searching, Analyzing Big Data?
I cannot find it on elastic.co page.


(Colin Goodheart-Smithe) #4

Unfortunately this question is still too vague. What do you mean by 'Big Data'? This term means different things to different people. What is the requirement that you are actually needing to solve? e.g. How to get data from Hadoop into Elasticsearch?


(Truong Van Linh) #5

I only want to summarize how it processing the Big Data (huge data sets: structure and non-structure data). If you felt this question is too vague, you can give me some LINKS or DOCUMENTS for consulting ^^.
Thanks you.


(Mark Harwood) #6

The general coping strategy for "Big Data" is to use the age-old tactic of "divide and conquer".

The dividing strategy is typically called "sharding" and involves putting only a subset of your overall data onto each machine. Queries can then run in parallel on many machines and then merge the results from many smaller pots of data faster than it would take to produce the same results if you used a single machine to search one very large pot of data.

Elasticsearch is not alone in adopting this approach. For general background here's a Wikipedia article[1]


(Truong Van Linh) #7

Thank for your helps, but i need more in formation in detail about how Elasticsearch process Big Data (Not focus on ES-Hadoop). Can anyone help me ? T.T


(Magnus Bäck) #8

Please, you must understand that the question is too vague. You will not get a useful answer as it stands. The Elasticsearch: The Definitive Guide book does a good job at explaining how Elasticsearch works. I suggest you start by reading it, and after a while you'll be able to ask more specific questions.


(Truong Van Linh) #9

Thanks you so much @magnusbaeck .
I understood what i need to do :smile:


(system) #10