# Moving from Sphinx to ES
**URL:** https://discuss.elastic.co/t/moving-from-sphinx-to-es/15094
**Category:** Elasticsearch
**Created:** [January 3, 2014, 5:08pm UTC](https://discuss.elastic.co/t/moving-from-sphinx-to-es/15094 "2014-01-03T17:08:55Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Brian\_Lovett](https://discuss.elastic.co/u/Brian_Lovett)
#### Post date: [January 3, 2014, 5:08pm UTC](https://discuss.elastic.co/t/moving-from-sphinx-to-es/15094/1 "2014-01-03T17:08:55Z")
Hi everyone, we're considering moving from Sphinx to Elastic Search, but I
want to make sure it is a good fit before rewriting our infrastructure.
Currently we have 20 dual octo core 2690 machines with 32GB of ram. We
handle about 2,000 queries per second with the existing setup, but we have
some pain points that I believe ES can help with.
1. Index rotation. Our index is about 30GB in size, but the nature of
Sphinx means that each week when our dataset is updated, we have to reindex
the entire dataset. Further, we have to do this on all 20 machines. We
currently reindex on one central machine, then rsync the data to the 20
main servers, then perform rolling restarts. Rsyncing 30GB of data to 20
machines takes too long, and it is only going to get worse as the number of
servers increases.
2. MySQL... Since Sphinx uses MySQL as a datastore, we are going to reach a
point where our database becomes a bottleneck. Our MySQL servers have no
problem handling the current load of 5,000 or so queries per second, but DB
servers are expensive to scale, and I would rather store the data in ES and
skip SQL completely.
My understanding is that ES shards the index across machines as they are
added. We have worked with cassandra in the past. The concept seems very
similar?
I know it's difficult to predict, but how is query performance with ES? Is
CPU or memory/IO the main bottleneck? We're moving to a new datacenter
where we will have 30 or so dual hex core machines but with 72GB of memory
each instead of the 32 we have in the current machines. Since ES shards the
index across machines, we should have no problem storing everything in
memory, so i'm guessing we would still be cpu bound (as we are with sphinx.
It isn't a problem, just wondering if I can expect the same).
Has anyone here moved a large scale sphinx cluster over to ES? Any gotcha's
that i'm overlooking? Any super easy migration plan you found that will cut
our development time from months down to days/weeks lol? I know, wishful
thinking.
Thanks in advance!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5934aa1c-84ee-4177-ad05-f449d4fe14f8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5934aa1c-84ee-4177-ad05-f449d4fe14f8%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@otisg](https://discuss.elastic.co/u/otisg)
#### Post date: [January 4, 2014, 3:49am UTC](https://discuss.elastic.co/t/moving-from-sphinx-to-es/15094/2 "2014-01-04T03:49:03Z")
HI Brian,
We helped a client successfully migrate from Sphinx to Solr last year.
Migration to ES should be similar to what we did for them for Solr.
Inline...
On Friday, January 3, 2014 12:08:55 PM UTC-5, Brian Lovett wrote:
> Hi everyone, we're considering moving from Sphinx to Elastic Search, but I
> want to make sure it is a good fit before rewriting our infrastructure.
>
> Currently we have 20 dual octo core 2690 machines with 32GB of ram. We
> handle about 2,000 queries per second with the existing setup, but we have
> some pain points that I believe ES can help with.
>
> 1. Index rotation. Our index is about 30GB in size, but the nature of
> Sphinx means that each week when our dataset is updated, we have to reindex
> the entire dataset. Further, we have to do this on all 20 machines. We
> currently reindex on one central machine, then rsync the data to the 20
> main servers, then perform rolling restarts. Rsyncing 30GB of data to 20
> machines takes too long, and it is only going to get worse as the number of
> servers increases.
Right, this sounds a bit old-style. With ES you'll be able to do
incremental indexing and you'll be able to forget about rsync.
> 1. MySQL... Since Sphinx uses MySQL as a datastore, we are going to reach
> a point where our database becomes a bottleneck. Our MySQL servers have no
> problem handling the current load of 5,000 or so queries per second, but DB
> servers are expensive to scale, and I would rather store the data in ES and
> skip SQL completely.
Right. Software like ES was built with sharding and replication in mind.
You still have to think about how to shard, replicate, query, etc. but it
will feel much more natural.
My understanding is that ES shards the index across machines as they are
> added. We have worked with cassandra in the past. The concept seems very
> similar?
Similar, but not quite the same. You tell ES how many shards you want to
split your index into when you create the index. ES currently cannot split
an existing shard and you currently can't change the number of shards later
on, but there are ways around that when/IF you end up needing that.
I know it's difficult to predict, but how is query performance with ES? Is
> CPU or memory/IO the main bottleneck? We're moving to a new datacenter
> where we will have 30 or so dual hex core machines but with 72GB of memory
> each instead of the 32 we have in the current machines. Since ES shards the
> index across machines, we should have no problem storing everything in
> memory, so i'm guessing we would still be cpu bound (as we are with sphinx.
> It isn't a problem, just wondering if I can expect the same).
Right, it's impossible to tell without a lot more details - query types.
complexity, cache utilization, etc. etc. But 30 dual hex core servers with
72 GB RAM everything should be in memory if your index ends up being more
or less the same size as in Sphinx.
> Has anyone here moved a large scale sphinx cluster over to ES? Any
> gotcha's that i'm overlooking? Any super easy migration plan you found that
> will cut our development time from months down to days/weeks lol? I know,
> wishful thinking.
In short: good decision. It shouldn't take months as far as ES work is
concerned.
> Thanks in advance!
## Otis
Performance Monitoring \* Log Analytics \* Search Analytics
Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ad81f216-bc40-49ac-b8fc-e2d02b454d83%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ad81f216-bc40-49ac-b8fc-e2d02b454d83%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:58am UTC](https://discuss.elastic.co/t/moving-from-sphinx-to-es/15094/3 "2017-07-06T01:58:22Z")