# Using ES as a cache layer **URL:** \ **Category:** Elasticsearch\ **Created:** [April 6, 2011, 7:15am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194 "2011-04-06T07:15:39Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Alexandre\_Heimburger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre_heimburger/32/2941_2.png) [@Alexandre\_Heimburger](https://discuss.elastic.co/u/Alexandre_Heimburger)\ **Post date:** [April 6, 2011, 7:15am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/1 "2011-04-06T07:15:39Z")
Hi, Do you think that an ES cluster can act as a cache layer ? More words : Data are stored in a SQL database, cached in a memcache cluster and indexed in a ES cluster. When an update occurs on an object, the 3 layers have to be updated. Is there performance gain or loss to use ES instead of the memcache for direct access to objects ? thx ---
**Author:** ![Alexandre\_Heimburger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre_heimburger/32/2941_2.png) [@Alexandre\_Heimburger](https://discuss.elastic.co/u/Alexandre_Heimburger)\ **Post date:** [April 6, 2011, 8:53am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/2 "2011-04-06T08:53:32Z")
I did some performance testing. 2529 users in DB --\> Get 2529 users in DB in 0.55709409713745 --\> Get 2529 users in Memcache in 0.32778406143188 --\> Get 2529 users in ES in 0.036777019500732 This makes me crazy. ES, 10x faster than memcache for this test. What do you think ? On 6 avr, 09:15, Alexandre Heimburger [alexheimbur...@gmail.com](mailto:alexheimbur...@gmail.com) wrote: > Hi, > > Do you think that an ES cluster can act as a cache layer ? > > More words : > > Data are stored in a SQL database, cached in a memcache cluster and indexed > in a ES cluster. When an update occurs on an object, the 3 layers have to be > updated. > > Is there performance gain or loss to use ES instead of the memcache for > direct access to objects ? > > thx ---
**Author:** ![Ridvan\_Gyundogan](https://avatars.discourse-cdn.com/v4/letter/r/e495f1/32.png) [@Ridvan\_Gyundogan](https://discuss.elastic.co/u/Ridvan_Gyundogan)\ **Post date:** [April 6, 2011, 9:03am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/3 "2011-04-06T09:03:48Z")
Hi Alexandre, we had simmilar ideas in our company, but our main concern was how do you do a join query, I don't know whether memcache supports this. --\> Get 2529 users in ES in 0.036777019500732 I suppose this is because this time is before they have been indexed. Regards On Wed, Apr 6, 2011 at 10:53 AM, alheim [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > I did some performance testing. > > 2529 users in DB > --\> Get 2529 users in DB in 0.55709409713745 > --\> Get 2529 users in Memcache in 0.32778406143188 > --\> Get 2529 users in ES in 0.036777019500732 > > This makes me crazy. ES, 10x faster than memcache for this test. > > What do you think ? > > On 6 avr, 09:15, Alexandre Heimburger [alexheimbur...@gmail.com](mailto:alexheimbur...@gmail.com) > wrote: > > > Hi, > > > > Do you think that an ES cluster can act as a cache layer ? > > > > More words : > > > > Data are stored in a SQL database, cached in a memcache cluster and > > indexed > > in a ES cluster. When an update occurs on an object, the 3 layers have to > > be > > updated. > > > > Is there performance gain or loss to use ES instead of the memcache for > > direct access to objects ? > > > > thx ---
**Author:** ![Ludovic\_Levesque](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ludovic_levesque/32/3031_2.png) [@Ludovic\_Levesque](https://discuss.elastic.co/u/Ludovic_Levesque)\ **Post date:** [April 6, 2011, 9:43am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/4 "2011-04-06T09:43:38Z")
On Wed, Apr 6, 2011 at 10:53 AM, alheim [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > I did some performance testing. > > 2529 users in DB > --\> Get 2529 users in DB in 0.55709409713745 > --\> Get 2529 users in Memcache in 0.32778406143188 > --\> Get 2529 users in ES in 0.036777019500732 > > This makes me crazy. ES, 10x faster than memcache for this test. > > What do you think ? Hi, can you briefly explain the benchmark ? Which DB software, which method to retrieve users (bulk, individual ?, for each three method) Having same performance between memcache and DB seems a little suspect at first sight. Ludo ---
**Author:** ![Alexandre\_Heimburger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre_heimburger/32/2941_2.png) [@Alexandre\_Heimburger](https://discuss.elastic.co/u/Alexandre_Heimburger)\ **Post date:** [April 6, 2011, 9:59am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/5 "2011-04-06T09:59:01Z")
No memcache does not support any query. It supports only a direct access by hash. 0.036777019500732 is the time to get the users once they have _been indexed_ On Wed, Apr 6, 2011 at 11:03 AM, Ridvan Gyundogan [ridvansg@gmail.com](mailto:ridvansg@gmail.com)wrote: > Hi Alexandre, > we had simmilar ideas in our company, but our main concern was how do you > do a join query, I don't know whether memcache supports this. > > --\> Get 2529 users in ES in 0.036777019500732 > > I suppose this is because this time is before they have been indexed. > > Regards > > On Wed, Apr 6, 2011 at 10:53 AM, alheim [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > > > I did some performance testing. > > > > 2529 users in DB > > --\> Get 2529 users in DB in 0.55709409713745 > > --\> Get 2529 users in Memcache in 0.32778406143188 > > --\> Get 2529 users in ES in 0.036777019500732 > > > > This makes me crazy. ES, 10x faster than memcache for this test. > > > > What do you think ? > > > > On 6 avr, 09:15, Alexandre Heimburger [alexheimbur...@gmail.com](mailto:alexheimbur...@gmail.com) > > wrote: > > > > > Hi, > > > > > > Do you think that an ES cluster can act as a cache layer ? > > > > > > More words : > > > > > > Data are stored in a SQL database, cached in a memcache cluster and > > > indexed > > > in a ES cluster. When an update occurs on an object, the 3 layers have > > > to be > > > updated. > > > > > > Is there performance gain or loss to use ES instead of the memcache for > > > direct access to objects ? > > > > > > thx -- Alexandre Heimburger R&D Manager blueKiwi Software tel : +33687880997 email : [ahb@bluekiwi-software.com](mailto:ahb@bluekiwi-software.com) adress : 93 rue Vieille du Temple, 75003 Paris What is blueKiwi? blueKiwi - the first Enterprise Social Software Suite in the world building professional networks on conversations and relationships - helps large organizations increase their productivity, foster innovations and boost people satisfaction. ---
**Author:** ![Ludovic\_Levesque](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ludovic_levesque/32/3031_2.png) [@Ludovic\_Levesque](https://discuss.elastic.co/u/Ludovic_Levesque)\ **Post date:** [April 6, 2011, 10:04am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/6 "2011-04-06T10:04:17Z")
Yes, but several libraries support get by array of keys. Do you use only one memcached server, or several ? same for ES: one node or several ? What was the size of documents ? On Wed, Apr 6, 2011 at 11:59 AM, Alexandre Heimburger [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > No memcache does not support any query. It supports only a direct access by > hash. > 0.036777019500732 is the time to get the users once they have been indexed > > On Wed, Apr 6, 2011 at 11:03 AM, Ridvan Gyundogan [ridvansg@gmail.com](mailto:ridvansg@gmail.com) > wrote: > > > Hi Alexandre, > > we had simmilar ideas in our company, but our main concern was how do you > > do a join query, I don't know whether memcache supports this. > > > > --\> Get 2529 users in ES in 0.036777019500732 > > > > I suppose this is because this time is before they have been indexed. > > > > Regards > > > > On Wed, Apr 6, 2011 at 10:53 AM, alheim [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > > > > > I did some performance testing. > > > > > > 2529 users in DB > > > --\> Get 2529 users in DB in 0.55709409713745 > > > --\> Get 2529 users in Memcache in 0.32778406143188 > > > --\> Get 2529 users in ES in 0.036777019500732 > > > > > > This makes me crazy. ES, 10x faster than memcache for this test. > > > > > > What do you think ? > > > > > > On 6 avr, 09:15, Alexandre Heimburger [alexheimbur...@gmail.com](mailto:alexheimbur...@gmail.com) > > > wrote: > > > > > > > Hi, > > > > > > > > Do you think that an ES cluster can act as a cache layer ? > > > > > > > > More words : > > > > > > > > Data are stored in a SQL database, cached in a memcache cluster and > > > > indexed > > > > in a ES cluster. When an update occurs on an object, the 3 layers have > > > > to be > > > > updated. > > > > > > > > Is there performance gain or loss to use ES instead of the memcache for > > > > direct access to objects ? > > > > > > > > thx > > -- > Alexandre Heimburger > R&D Manager > blueKiwi Software > tel : +33687880997 > email : [ahb@bluekiwi-software.com](mailto:ahb@bluekiwi-software.com) > adress : 93 rue Vieille du Temple, 75003 Paris > > What is blueKiwi? blueKiwi - the first Enterprise Social Software Suite in > the world building professional networks on conversations and relationships > > - helps large organizations increase their productivity, foster innovations > and boost people satisfaction. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 6, 2011, 10:14am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/7 "2011-04-06T10:14:18Z")
Its strange that ES will be faster then memcached on a single GET by key API.... , it is really fast, but memcached should be faster. Of course, this depends on so many things, starting with how you talk to ES and talk to memcached. On Wednesday, April 6, 2011 at 1:04 PM, Ludovic Levesque wrote: > Yes, but several libraries support get by array of keys. Do you use > only one memcached server, or several ? same for ES: one node or > several ? > > What was the size of documents ? > > On Wed, Apr 6, 2011 at 11:59 AM, Alexandre Heimburger > [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > > > No memcache does not support any query. It supports only a direct access by > > hash. > > 0.036777019500732 is the time to get the users once they have been indexed > > > > On Wed, Apr 6, 2011 at 11:03 AM, Ridvan Gyundogan [ridvansg@gmail.com](mailto:ridvansg@gmail.com) > > wrote: > > > > > Hi Alexandre, > > > we had simmilar ideas in our company, but our main concern was how do you > > > do a join query, I don't know whether memcache supports this. > > > > > > --\> Get 2529 users in ES in 0.036777019500732 > > > > > > I suppose this is because this time is before they have been indexed. > > > > > > Regards > > > > > > On Wed, Apr 6, 2011 at 10:53 AM, alheim [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > > > > > > > I did some performance testing. > > > > > > > > 2529 users in DB > > > > --\> Get 2529 users in DB in 0.55709409713745 > > > > --\> Get 2529 users in Memcache in 0.32778406143188 > > > > --\> Get 2529 users in ES in 0.036777019500732 > > > > > > > > This makes me crazy. ES, 10x faster than memcache for this test. > > > > > > > > What do you think ? > > > > > > > > On 6 avr, 09:15, Alexandre Heimburger [alexheimbur...@gmail.com](mailto:alexheimbur...@gmail.com) > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > Do you think that an ES cluster can act as a cache layer ? > > > > > > > > > > More words : > > > > > > > > > > Data are stored in a SQL database, cached in a memcache cluster and > > > > > indexed > > > > > in a ES cluster. When an update occurs on an object, the 3 layers have > > > > > to be > > > > > updated. > > > > > > > > > > Is there performance gain or loss to use ES instead of the memcache for > > > > > direct access to objects ? > > > > > > > > > > thx > > > > -- > > Alexandre Heimburger > > R&D Manager > > blueKiwi Software > > tel : +33687880997 > > email : [ahb@bluekiwi-software.com](mailto:ahb@bluekiwi-software.com) > > adress : 93 rue Vieille du Temple, 75003 Paris > > > > What is blueKiwi? blueKiwi - the first Enterprise Social Software Suite in > > the world building professional networks on conversations and relationships > > > > - helps large organizations increase their productivity, foster innovations > > and boost people satisfaction. ---
**Author:** ![Joaquin\_Cuenca\_Abela](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@Joaquin\_Cuenca\_Abela](https://discuss.elastic.co/u/Joaquin_Cuenca_Abela)\ **Post date:** [April 6, 2011, 10:16am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/8 "2011-04-06T10:16:51Z")
Hi, can you gist the benchmark? I suspect you're doing 2526 single gets to memcache instead of a multiget, and thus you're only testing the speed of your OS doing context switches or the speed of your network. On Wed, Apr 6, 2011 at 10:53 AM, alheim [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > I did some performance testing. > > 2529 users in DB > --\> Get 2529 users in DB in 0.55709409713745 > --\> Get 2529 users in Memcache in 0.32778406143188 > --\> Get 2529 users in ES in 0.036777019500732 > > This makes me crazy. ES, 10x faster than memcache for this test. > > What do you think ? > > On 6 avr, 09:15, Alexandre Heimburger [alexheimbur...@gmail.com](mailto:alexheimbur...@gmail.com) > wrote: > > > Hi, > > > > Do you think that an ES cluster can act as a cache layer ? > > > > More words : > > > > Data are stored in a SQL database, cached in a memcache cluster and indexed > > in a ES cluster. When an update occurs on an object, the 3 layers have to be > > updated. > > > > Is there performance gain or loss to use ES instead of the memcache for > > direct access to objects ? > > > > thx -- Joaquin Cuenca Abela -- [presspeople.com](http://presspeople.com): Fuentes de prensa y comunicados ---
**Author:** ![Alexandre\_Heimburger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre_heimburger/32/2941_2.png) [@Alexandre\_Heimburger](https://discuss.elastic.co/u/Alexandre_Heimburger)\ **Post date:** [April 6, 2011, 3:06pm UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/9 "2011-04-06T15:06:58Z")
No I'm not doing single gets...please. The thing is that some documents are larger than 1MB, which is larger than the max size of a memcached bucket. _So serialization is done._ This may be not fair but the goal of this quick benchmark was not to test the communication layer of each software but their use as a cache backend. I use one ES node, one memcache and one mysql server. Both are on on the same lan. On Wed, Apr 6, 2011 at 12:16 PM, Joaquin Cuenca Abela \< [joaquin@cuencaabela.com](mailto:joaquin@cuencaabela.com)\> wrote: > Hi, > > can you gist the benchmark? I suspect you're doing 2526 single gets to > memcache instead of a multiget, and thus you're only testing the speed > of your OS doing context switches or the speed of your network. > > On Wed, Apr 6, 2011 at 10:53 AM, alheim [alexheimburger@gmail.com](mailto:alexheimburger@gmail.com) wrote: > > > I did some performance testing. > > > > 2529 users in DB > > --\> Get 2529 users in DB in 0.55709409713745 > > --\> Get 2529 users in Memcache in 0.32778406143188 > > --\> Get 2529 users in ES in 0.036777019500732 > > > > This makes me crazy. ES, 10x faster than memcache for this test. > > > > What do you think ? > > > > On 6 avr, 09:15, Alexandre Heimburger [alexheimbur...@gmail.com](mailto:alexheimbur...@gmail.com) > > wrote: > > > > > Hi, > > > > > > Do you think that an ES cluster can act as a cache layer ? > > > > > > More words : > > > > > > Data are stored in a SQL database, cached in a memcache cluster and > > > indexed > > > in a ES cluster. When an update occurs on an object, the 3 layers have > > > to be > > > updated. > > > > > > Is there performance gain or loss to use ES instead of the memcache for > > > direct access to objects ? > > > > > > thx > > -- > Joaquin Cuenca Abela -- [presspeople.com](http://presspeople.com): Fuentes de prensa y comunicados -- Alexandre Heimburger R&D Manager blueKiwi Software tel : +33687880997 email : [ahb@bluekiwi-software.com](mailto:ahb@bluekiwi-software.com) adress : 93 rue Vieille du Temple, 75003 Paris What is blueKiwi? blueKiwi - the first Enterprise Social Software Suite in the world building professional networks on conversations and relationships - helps large organizations increase their productivity, foster innovations and boost people satisfaction. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:08am UTC](https://discuss.elastic.co/t/using-es-as-a-cache-layer/4194/10 "2017-07-06T04:08:47Z")