Hi,
Do you think that an ES cluster can act as a cache layer ?
More words :
Data are stored in a SQL database, cached in a memcache cluster and indexed
in a ES cluster. When an update occurs on an object, the 3 layers have to be
updated.
Is there performance gain or loss to use ES instead of the memcache for
direct access to objects ?
thx
I did some performance testing.
2529 users in DB
--> Get 2529 users in DB in 0.55709409713745
--> Get 2529 users in Memcache in 0.32778406143188
--> Get 2529 users in ES in 0.036777019500732
This makes me crazy. ES, 10x faster than memcache for this test.
What do you think ?
On 6 avr, 09:15, Alexandre Heimburger alexheimbur...@gmail.com
wrote:
Hi,
Do you think that an ES cluster can act as a cache layer ?
More words :
Data are stored in a SQL database, cached in a memcache cluster and indexed
in a ES cluster. When an update occurs on an object, the 3 layers have to be
updated.Is there performance gain or loss to use ES instead of the memcache for
direct access to objects ?thx
Hi Alexandre,
we had simmilar ideas in our company, but our main concern was how do you do
a join query, I don't know whether memcache supports this.
--> Get 2529 users in ES in 0.036777019500732
I suppose this is because this time is before they have been indexed.
Regards
On Wed, Apr 6, 2011 at 10:53 AM, alheim alexheimburger@gmail.com wrote:
I did some performance testing.
2529 users in DB
--> Get 2529 users in DB in 0.55709409713745
--> Get 2529 users in Memcache in 0.32778406143188
--> Get 2529 users in ES in 0.036777019500732This makes me crazy. ES, 10x faster than memcache for this test.
What do you think ?
On 6 avr, 09:15, Alexandre Heimburger alexheimbur...@gmail.com
wrote:Hi,
Do you think that an ES cluster can act as a cache layer ?
More words :
Data are stored in a SQL database, cached in a memcache cluster and
indexed
in a ES cluster. When an update occurs on an object, the 3 layers have to
be
updated.Is there performance gain or loss to use ES instead of the memcache for
direct access to objects ?thx
On Wed, Apr 6, 2011 at 10:53 AM, alheim alexheimburger@gmail.com wrote:
I did some performance testing.
2529 users in DB
--> Get 2529 users in DB in 0.55709409713745
--> Get 2529 users in Memcache in 0.32778406143188
--> Get 2529 users in ES in 0.036777019500732This makes me crazy. ES, 10x faster than memcache for this test.
What do you think ?
Hi,
can you briefly explain the benchmark ?
Which DB software, which method to retrieve users (bulk, individual ?,
for each three method)
Having same performance between memcache and DB seems a little suspect
at first sight.
Ludo
No memcache does not support any query. It supports only a direct access by
hash.
0.036777019500732 is the time to get the users once they have been indexed
On Wed, Apr 6, 2011 at 11:03 AM, Ridvan Gyundogan ridvansg@gmail.comwrote:
Hi Alexandre,
we had simmilar ideas in our company, but our main concern was how do you
do a join query, I don't know whether memcache supports this.--> Get 2529 users in ES in 0.036777019500732
I suppose this is because this time is before they have been indexed.
Regards
On Wed, Apr 6, 2011 at 10:53 AM, alheim alexheimburger@gmail.com wrote:
I did some performance testing.
2529 users in DB
--> Get 2529 users in DB in 0.55709409713745
--> Get 2529 users in Memcache in 0.32778406143188
--> Get 2529 users in ES in 0.036777019500732This makes me crazy. ES, 10x faster than memcache for this test.
What do you think ?
On 6 avr, 09:15, Alexandre Heimburger alexheimbur...@gmail.com
wrote:Hi,
Do you think that an ES cluster can act as a cache layer ?
More words :
Data are stored in a SQL database, cached in a memcache cluster and
indexed
in a ES cluster. When an update occurs on an object, the 3 layers have
to be
updated.Is there performance gain or loss to use ES instead of the memcache for
direct access to objects ?thx
--
Alexandre Heimburger
R&D Manager
blueKiwi Software
tel : +33687880997
email : ahb@bluekiwi-software.com
adress : 93 rue Vieille du Temple, 75003 Paris
What is blueKiwi? blueKiwi - the first Enterprise Social Software Suite in
the world building professional networks on conversations and relationships
Yes, but several libraries support get by array of keys. Do you use
only one memcached server, or several ? same for ES: one node or
several ?
What was the size of documents ?
On Wed, Apr 6, 2011 at 11:59 AM, Alexandre Heimburger
alexheimburger@gmail.com wrote:
No memcache does not support any query. It supports only a direct access by
hash.
0.036777019500732 is the time to get the users once they have been indexedOn Wed, Apr 6, 2011 at 11:03 AM, Ridvan Gyundogan ridvansg@gmail.com
wrote:Hi Alexandre,
we had simmilar ideas in our company, but our main concern was how do you
do a join query, I don't know whether memcache supports this.--> Get 2529 users in ES in 0.036777019500732
I suppose this is because this time is before they have been indexed.
Regards
On Wed, Apr 6, 2011 at 10:53 AM, alheim alexheimburger@gmail.com wrote:
I did some performance testing.
2529 users in DB
--> Get 2529 users in DB in 0.55709409713745
--> Get 2529 users in Memcache in 0.32778406143188
--> Get 2529 users in ES in 0.036777019500732This makes me crazy. ES, 10x faster than memcache for this test.
What do you think ?
On 6 avr, 09:15, Alexandre Heimburger alexheimbur...@gmail.com
wrote:Hi,
Do you think that an ES cluster can act as a cache layer ?
More words :
Data are stored in a SQL database, cached in a memcache cluster and
indexed
in a ES cluster. When an update occurs on an object, the 3 layers have
to be
updated.Is there performance gain or loss to use ES instead of the memcache for
direct access to objects ?thx
--
Alexandre Heimburger
R&D Manager
blueKiwi Software
tel : +33687880997
email : ahb@bluekiwi-software.com
adress : 93 rue Vieille du Temple, 75003 ParisWhat is blueKiwi? blueKiwi - the first Enterprise Social Software Suite in
the world building professional networks on conversations and relationships
- helps large organizations increase their productivity, foster innovations
and boost people satisfaction.
Its strange that ES will be faster then memcached on a single GET by key API.... , it is really fast, but memcached should be faster. Of course, this depends on so many things, starting with how you talk to ES and talk to memcached.
On Wednesday, April 6, 2011 at 1:04 PM, Ludovic Levesque wrote:
Yes, but several libraries support get by array of keys. Do you use
only one memcached server, or several ? same for ES: one node or
several ?What was the size of documents ?
On Wed, Apr 6, 2011 at 11:59 AM, Alexandre Heimburger
alexheimburger@gmail.com wrote:No memcache does not support any query. It supports only a direct access by
hash.
0.036777019500732 is the time to get the users once they have been indexedOn Wed, Apr 6, 2011 at 11:03 AM, Ridvan Gyundogan ridvansg@gmail.com
wrote:Hi Alexandre,
we had simmilar ideas in our company, but our main concern was how do you
do a join query, I don't know whether memcache supports this.--> Get 2529 users in ES in 0.036777019500732
I suppose this is because this time is before they have been indexed.
Regards
On Wed, Apr 6, 2011 at 10:53 AM, alheim alexheimburger@gmail.com wrote:
I did some performance testing.
2529 users in DB
--> Get 2529 users in DB in 0.55709409713745
--> Get 2529 users in Memcache in 0.32778406143188
--> Get 2529 users in ES in 0.036777019500732This makes me crazy. ES, 10x faster than memcache for this test.
What do you think ?
On 6 avr, 09:15, Alexandre Heimburger alexheimbur...@gmail.com
wrote:Hi,
Do you think that an ES cluster can act as a cache layer ?
More words :
Data are stored in a SQL database, cached in a memcache cluster and
indexed
in a ES cluster. When an update occurs on an object, the 3 layers have
to be
updated.Is there performance gain or loss to use ES instead of the memcache for
direct access to objects ?thx
--
Alexandre Heimburger
R&D Manager
blueKiwi Software
tel : +33687880997
email : ahb@bluekiwi-software.com
adress : 93 rue Vieille du Temple, 75003 ParisWhat is blueKiwi? blueKiwi - the first Enterprise Social Software Suite in
the world building professional networks on conversations and relationships
- helps large organizations increase their productivity, foster innovations
and boost people satisfaction.
Hi,
can you gist the benchmark? I suspect you're doing 2526 single gets to
memcache instead of a multiget, and thus you're only testing the speed
of your OS doing context switches or the speed of your network.
On Wed, Apr 6, 2011 at 10:53 AM, alheim alexheimburger@gmail.com wrote:
I did some performance testing.
2529 users in DB
--> Get 2529 users in DB in 0.55709409713745
--> Get 2529 users in Memcache in 0.32778406143188
--> Get 2529 users in ES in 0.036777019500732This makes me crazy. ES, 10x faster than memcache for this test.
What do you think ?
On 6 avr, 09:15, Alexandre Heimburger alexheimbur...@gmail.com
wrote:Hi,
Do you think that an ES cluster can act as a cache layer ?
More words :
Data are stored in a SQL database, cached in a memcache cluster and indexed
in a ES cluster. When an update occurs on an object, the 3 layers have to be
updated.Is there performance gain or loss to use ES instead of the memcache for
direct access to objects ?thx
--
Joaquin Cuenca Abela -- presspeople.com: Fuentes de prensa y comunicados
No I'm not doing single gets...please.
The thing is that some documents are larger than 1MB, which is larger than
the max size of a memcached bucket. So serialization is done. This may be
not fair but the goal of this quick benchmark was not to test the
communication layer of each software but their use as a cache backend.
I use one ES node, one memcache and one mysql server. Both are on on the
same lan.
On Wed, Apr 6, 2011 at 12:16 PM, Joaquin Cuenca Abela <
joaquin@cuencaabela.com> wrote:
Hi,
can you gist the benchmark? I suspect you're doing 2526 single gets to
memcache instead of a multiget, and thus you're only testing the speed
of your OS doing context switches or the speed of your network.On Wed, Apr 6, 2011 at 10:53 AM, alheim alexheimburger@gmail.com wrote:
I did some performance testing.
2529 users in DB
--> Get 2529 users in DB in 0.55709409713745
--> Get 2529 users in Memcache in 0.32778406143188
--> Get 2529 users in ES in 0.036777019500732This makes me crazy. ES, 10x faster than memcache for this test.
What do you think ?
On 6 avr, 09:15, Alexandre Heimburger alexheimbur...@gmail.com
wrote:Hi,
Do you think that an ES cluster can act as a cache layer ?
More words :
Data are stored in a SQL database, cached in a memcache cluster and
indexed
in a ES cluster. When an update occurs on an object, the 3 layers have
to be
updated.Is there performance gain or loss to use ES instead of the memcache for
direct access to objects ?thx
--
Joaquin Cuenca Abela -- presspeople.com: Fuentes de prensa y comunicados
--
Alexandre Heimburger
R&D Manager
blueKiwi Software
tel : +33687880997
email : ahb@bluekiwi-software.com
adress : 93 rue Vieille du Temple, 75003 Paris
What is blueKiwi? blueKiwi - the first Enterprise Social Software Suite in
the world building professional networks on conversations and relationships
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.