Configuration for High Volume, Real-time searching


(RK) #1

Hi,

I'm trying to configure ElasticSearch for a High Volume data with
(near) real-time searching. The data is hundreds of millions
documents, with Tens of documents being loaded every second (as new
docs are made available).

Can someone provide me configuration and/or other optimization
techniques so that search can be practically usable for 100s of users
online?

Acceptable search time is 2-3 seconds, with search on anywhere
between 1-10 fields.

Thanks!
-RK


(Otis Gospodnetić) #2

RK,

Are you seeing issues right now? If not, I suggest you simply try
using ES the best you can and then, if you have issues, email the list
and ask for help or go to ES experts to provide assistance.

Otis

On Feb 9, 9:12 am, RK rk4...@gmail.com wrote:

Hi,

I'm trying to configure ElasticSearch for a High Volume data with
(near) real-time searching. The data is hundreds of millions
documents, with Tens of documents being loaded every second (as new
docs are made available).

Can someone provide me configuration and/or other optimization
techniques so that search can be practically usable for 100s of users
online?

Acceptable search time is 2-3 seconds, with search on anywhere
between 1-10 fields.

Thanks!
-RK


(Shay Banon) #3

One more thing that I would suggest is to make sure to think about the indexed data and how it looks like. See more here: https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/data$20flow/elasticsearch/49q-_AgQCp8/MRol0t9asEcJ.

On Friday, February 10, 2012 at 5:35 AM, Otis Gospodnetic wrote:

RK,

Are you seeing issues right now? If not, I suggest you simply try
using ES the best you can and then, if you have issues, email the list
and ask for help or go to ES experts to provide assistance.

Otis

http://sematext.com/

On Feb 9, 9:12 am, RK <rk4...@gmail.com (http://gmail.com)> wrote:

Hi,

I'm trying to configure ElasticSearch for a High Volume data with
(near) real-time searching. The data is hundreds of millions
documents, with Tens of documents being loaded every second (as new
docs are made available).

Can someone provide me configuration and/or other optimization
techniques so that search can be practically usable for 100s of users
online?

Acceptable search time is 2-3 seconds, with search on anywhere
between 1-10 fields.

Thanks!
-RK


(system) #4