Current or planned usage in production


(David Jensen-2) #1

I'm posting this again because I got only one response. It was a good
response but still only one data point.

Who has Elastic Search in production or plans to deploy it into
production soon? Any Web 2.0 companies?

How many documents do you have in the index? How large are each
document? How many nodes are you running?

Thanks,
David Jensen


(Grant Rodgers) #2

GoTime has ElasticSearch in production. We are currently using it to
index pageview analytics as a trial while we prepare other code that
uses it.

Our current index has 6.5 million documents with around 25 fields
each. Future plans are for multiple indices each with 4-5 million
complex nested documents.

We have 3 c1.xlarge instances on ec2 running both cassandra and
elasticsearch. Cpu usage is very low but load stays around the 6-8
range, so we are using the 8-core instances for now. We are using the
S3 gateway but haven't had any restore scenarios yet. In our testing,
recovery of large indices from S3 has not been promising, so we are
looking forward to the reuse of work directories feature that is being
worked on. To be fair, the recovery testing was done with 0.7.2, and I
understand the gateway format has changed since then.

Another feature that we're looking forward to is the enhanced facet
support, especially histogram facets.

Overall I've been very impressed with ElasticSearch and the pace of
development. Keep up the good work Shay!

Grant

On Jun 28, 11:52 am, David Jensen djense...@gmail.com wrote:

I'm posting this again because I got only one response. It was a good
response but still only one data point.

Who has Elastic Search in production or plans to deploy it into
production soon? Any Web 2.0 companies?

How many documents do you have in the index? How large are each
document? How many nodes are you running?

Thanks,
David Jensen


(Shay Banon) #3

Hi Grant (and all),

Is there a chance that you can add it to the wiki page:
http://wiki.github.com/elasticsearch/elasticsearch/powered-by? The more
people using and stating that they use it, the better other users will feel
about it.

Currently, it only lists a link to the site / company that uses it, but I
think the below description you gave is great info as well, so maybe, if you
are going to update the wiki, also add the description?

-shay.banon

On Tue, Jun 29, 2010 at 12:39 AM, Grant Rodgers grantr@gmail.com wrote:

GoTime has ElasticSearch in production. We are currently using it to
index pageview analytics as a trial while we prepare other code that
uses it.

Our current index has 6.5 million documents with around 25 fields
each. Future plans are for multiple indices each with 4-5 million
complex nested documents.

We have 3 c1.xlarge instances on ec2 running both cassandra and
elasticsearch. Cpu usage is very low but load stays around the 6-8
range, so we are using the 8-core instances for now. We are using the
S3 gateway but haven't had any restore scenarios yet. In our testing,
recovery of large indices from S3 has not been promising, so we are
looking forward to the reuse of work directories feature that is being
worked on. To be fair, the recovery testing was done with 0.7.2, and I
understand the gateway format has changed since then.

Another feature that we're looking forward to is the enhanced facet
support, especially histogram facets.

Overall I've been very impressed with ElasticSearch and the pace of
development. Keep up the good work Shay!

Grant

On Jun 28, 11:52 am, David Jensen djense...@gmail.com wrote:

I'm posting this again because I got only one response. It was a good
response but still only one data point.

Who has Elastic Search in production or plans to deploy it into
production soon? Any Web 2.0 companies?

How many documents do you have in the index? How large are each
document? How many nodes are you running?

Thanks,
David Jensen


(Grant Rodgers) #4

Added!

On Jun 28, 4:01 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Hi Grant (and all),

Is there a chance that you can add it to the wiki page:http://wiki.github.com/elasticsearch/elasticsearch/powered-by?The more
people using and stating that they use it, the better other users will feel
about it.

Currently, it only lists a link to the site / company that uses it, but I
think the below description you gave is great info as well, so maybe, if you
are going to update the wiki, also add the description?

-shay.banon

On Tue, Jun 29, 2010 at 12:39 AM, Grant Rodgers gra...@gmail.com wrote:

GoTime has ElasticSearch in production. We are currently using it to
index pageview analytics as a trial while we prepare other code that
uses it.

Our current index has 6.5 million documents with around 25 fields
each. Future plans are for multiple indices each with 4-5 million
complex nested documents.

We have 3 c1.xlarge instances on ec2 running both cassandra and
elasticsearch. Cpu usage is very low but load stays around the 6-8
range, so we are using the 8-core instances for now. We are using the
S3 gateway but haven't had any restore scenarios yet. In our testing,
recovery of large indices from S3 has not been promising, so we are
looking forward to the reuse of work directories feature that is being
worked on. To be fair, the recovery testing was done with 0.7.2, and I
understand the gateway format has changed since then.

Another feature that we're looking forward to is the enhanced facet
support, especially histogram facets.

Overall I've been very impressed with ElasticSearch and the pace of
development. Keep up the good work Shay!

Grant

On Jun 28, 11:52 am, David Jensen djense...@gmail.com wrote:

I'm posting this again because I got only one response. It was a good
response but still only one data point.

Who has Elastic Search in production or plans to deploy it into
production soon? Any Web 2.0 companies?

How many documents do you have in the index? How large are each
document? How many nodes are you running?

Thanks,
David Jensen


(David Jensen-2) #5

I'm going to bump this thread because every time I post this topic I
get one new person saying they use it.

I've already started testing ElasticSearch and I'm going to try to
index 30M+ documents. We'll see how it goes.

On Jun 28, 6:07 pm, Grant Rodgers gra...@gmail.com wrote:

Added!

On Jun 28, 4:01 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Hi Grant (and all),

Is there a chance that you can add it to the wiki page:http://wiki.github.com/elasticsearch/elasticsearch/powered-by?Themore
people using and stating that they use it, the better other users will feel
about it.

Currently, it only lists a link to the site / company that uses it, but I
think the below description you gave is great info as well, so maybe, if you
are going to update the wiki, also add the description?

-shay.banon

On Tue, Jun 29, 2010 at 12:39 AM, Grant Rodgers gra...@gmail.com wrote:

GoTime has ElasticSearch in production. We are currently using it to
index pageview analytics as a trial while we prepare other code that
uses it.

Our current index has 6.5 million documents with around 25 fields
each. Future plans are for multiple indices each with 4-5 million
complex nested documents.

We have 3 c1.xlarge instances on ec2 running both cassandra and
elasticsearch. Cpu usage is very low but load stays around the 6-8
range, so we are using the 8-core instances for now. We are using the
S3 gateway but haven't had any restore scenarios yet. In our testing,
recovery of large indices from S3 has not been promising, so we are
looking forward to the reuse of work directories feature that is being
worked on. To be fair, the recovery testing was done with 0.7.2, and I
understand the gateway format has changed since then.

Another feature that we're looking forward to is the enhanced facet
support, especially histogram facets.

Overall I've been very impressed with ElasticSearch and the pace of
development. Keep up the good work Shay!

Grant

On Jun 28, 11:52 am, David Jensen djense...@gmail.com wrote:

I'm posting this again because I got only one response. It was a good
response but still only one data point.

Who has Elastic Search in production or plans to deploy it into
production soon? Any Web 2.0 companies?

How many documents do you have in the index? How large are each
document? How many nodes are you running?

Thanks,
David Jensen


(Otis Gospodnetić) #6

Keep bumping it :wink:
We are about to use ES in a "multi-hundred-millions" docs
environment. :slight_smile:

Otis

Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/

On Jul 6, 7:43 pm, David Jensen djense...@gmail.com wrote:

I'm going to bump this thread because every time I post this topic I
get one new person saying they use it.

I've already started testing ElasticSearch and I'm going to try to
index 30M+ documents. We'll see how it goes.

On Jun 28, 6:07 pm, Grant Rodgers gra...@gmail.com wrote:

Added!

On Jun 28, 4:01 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Hi Grant (and all),

Is there a chance that you can add it to the wiki page:http://wiki.github.com/elasticsearch/elasticsearch/powered-by?Themore
people using and stating that they use it, the better other users will feel
about it.

Currently, it only lists a link to the site / company that uses it, but I
think the below description you gave is great info as well, so maybe, if you
are going to update the wiki, also add the description?

-shay.banon

On Tue, Jun 29, 2010 at 12:39 AM, Grant Rodgers gra...@gmail.com wrote:

GoTime has ElasticSearch in production. We are currently using it to
index pageview analytics as a trial while we prepare other code that
uses it.

Our current index has 6.5 million documents with around 25 fields
each. Future plans are for multiple indices each with 4-5 million
complex nested documents.

We have 3 c1.xlarge instances on ec2 running both cassandra and
elasticsearch. Cpu usage is very low but load stays around the 6-8
range, so we are using the 8-core instances for now. We are using the
S3 gateway but haven't had any restore scenarios yet. In our testing,
recovery of large indices from S3 has not been promising, so we are
looking forward to the reuse of work directories feature that is being
worked on. To be fair, the recovery testing was done with 0.7.2, and I
understand the gateway format has changed since then.

Another feature that we're looking forward to is the enhanced facet
support, especially histogram facets.

Overall I've been very impressed with ElasticSearch and the pace of
development. Keep up the good work Shay!

Grant

On Jun 28, 11:52 am, David Jensen djense...@gmail.com wrote:

I'm posting this again because I got only one response. It was a good
response but still only one data point.

Who has Elastic Search in production or plans to deploy it into
production soon? Any Web 2.0 companies?

How many documents do you have in the index? How large are each
document? How many nodes are you running?

Thanks,
David Jensen


(David Jensen-2) #7

Bump.

Who else is using ElasticSearch in production or soon in production.
Post it on the wiki ...
http://wiki.github.com/elasticsearch/elasticsearch/powered-by?

On Jul 7, 1:35 pm, Otis otis.gospodne...@gmail.com wrote:

Keep bumping it :wink:
We are about to use ES in a "multi-hundred-millions" docs
environment. :slight_smile:

Otis

Sematext ::http://sematext.com/:: Solr - Lucene - Nutch
Lucene ecosystem search ::http://search-lucene.com/

On Jul 6, 7:43 pm, David Jensen djense...@gmail.com wrote:

I'm going to bump this thread because every time I post this topic I
get one new person saying they use it.

I've already started testing ElasticSearch and I'm going to try to
index 30M+ documents. We'll see how it goes.

On Jun 28, 6:07 pm, Grant Rodgers gra...@gmail.com wrote:

Added!

On Jun 28, 4:01 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Hi Grant (and all),

Is there a chance that you can add it to the wiki page:http://wiki.github.com/elasticsearch/elasticsearch/powered-by?Themore
people using and stating that they use it, the better other users will feel
about it.

Currently, it only lists a link to the site / company that uses it, but I
think the below description you gave is great info as well, so maybe, if you
are going to update the wiki, also add the description?

-shay.banon

On Tue, Jun 29, 2010 at 12:39 AM, Grant Rodgers gra...@gmail.com wrote:

GoTime has ElasticSearch in production. We are currently using it to
index pageview analytics as a trial while we prepare other code that
uses it.

Our current index has 6.5 million documents with around 25 fields
each. Future plans are for multiple indices each with 4-5 million
complex nested documents.

We have 3 c1.xlarge instances on ec2 running both cassandra and
elasticsearch. Cpu usage is very low but load stays around the 6-8
range, so we are using the 8-core instances for now. We are using the
S3 gateway but haven't had any restore scenarios yet. In our testing,
recovery of large indices from S3 has not been promising, so we are
looking forward to the reuse of work directories feature that is being
worked on. To be fair, the recovery testing was done with 0.7.2, and I
understand the gateway format has changed since then.

Another feature that we're looking forward to is the enhanced facet
support, especially histogram facets.

Overall I've been very impressed with ElasticSearch and the pace of
development. Keep up the good work Shay!

Grant

On Jun 28, 11:52 am, David Jensen djense...@gmail.com wrote:

I'm posting this again because I got only one response. It was a good
response but still only one data point.

Who has Elastic Search in production or plans to deploy it into
production soon? Any Web 2.0 companies?

How many documents do you have in the index? How large are each
document? How many nodes are you running?

Thanks,
David Jensen


(system) #8