Replacing redis with elasticsearch (GET query speed, counters and lists)?

Elastic Stack Elasticsearch
1

I am currently considering replacing redis with elasticsearch to
simplify the stack and have run into the following 3 use cases:

  1. I am currently using redis for caching. Redis is benchmarked at
    around 100,000 GETs per second when querying by key (http://redis.io/
    topics/benchmarks). How does ES stack in terms simple GET by _id
    performance (assuming real-time get)?

  2. I am currently using redis to store counters (EG: 'foo = foo +
    1280' types of commands). Is there any way to implement the concept of
    counters in ES with support atomic updates (maybe using the script
    command)?

  3. I am currently using redis to store lists for queuing use cases (in
    which items are added to a list and then popped off). Can something
    like that be effectively implemented in ES?

Thanks and keep up the great work!

1 Like
2

In all three points you mention: GET, INCRBY, atomic LPOP, I'd
consider Redis superior to Elasticsearch.

I wouldn't get rid of Redis just to "simplify the stack". There's
always something where Redis fits as a glove.

(And I love Elasticsearch as much as Redis, possible more :slight_smile:

Karel

On Oct 16, 11:20 pm, datadev n...@adinfocenter.com wrote:

I am currently considering replacing redis with elasticsearch to
simplify the stack and have run into the following 3 use cases:

  1. I am currently using redis for caching. Redis is benchmarked at
    around 100,000 GETs per second when querying by key (http://redis.io/
    topics/benchmarks). How does ES stack in terms simple GET by _id
    performance (assuming real-time get)?

  2. I am currently using redis to store counters (EG: 'foo = foo +
    1280' types of commands). Is there any way to implement the concept of
    counters in ES with support atomic updates (maybe using the script
    command)?

  3. I am currently using redis to store lists for queuing use cases (in
    which items are added to a list and then popped off). Can something
    like that be effectively implemented in ES?

Thanks and keep up the great work!

3

the third one could be handled with one index (instead of the list)
and using the _version field to avoid problems (can be used to solve
point 2 as well) when adding or deleting items and sort against a
timestamp or similar.

the other mentioned points are probably better implemented with redis
(there are no transactions in ES) ... you could try point 1 with ES:
but even with a ids-request

100k req/sec seems to be high :slight_smile: of course all depends on a lot things
e.g. putting it all into RAM ... let me know how it compares :slight_smile: !

On 17 Okt., 08:08, karmi karel.mina...@gmail.com wrote:

In all three points you mention: GET, INCRBY, atomic LPOP, I'd
consider Redis superior to Elasticsearch.

I wouldn't get rid of Redis just to "simplify the stack". There's
always something where Redis fits as a glove.

(And I love Elasticsearch as much as Redis, possible more :slight_smile:

Karel

On Oct 16, 11:20 pm, datadev n...@adinfocenter.com wrote:

I am currently considering replacing redis with elasticsearch to
simplify the stack and have run into the following 3 use cases:

  1. I am currently using redis for caching. Redis is benchmarked at
    around 100,000 GETs per second when querying by key (http://redis.io/
    topics/benchmarks). How does ES stack in terms simple GET by _id
    performance (assuming real-time get)?
  1. I am currently using redis to store counters (EG: 'foo = foo +
    1280' types of commands). Is there any way to implement the concept of
    counters in ES with support atomic updates (maybe using the script
    command)?
  1. I am currently using redis to store lists for queuing use cases (in
    which items are added to a list and then popped off). Can something
    like that be effectively implemented in ES?

Thanks and keep up the great work!

4

Yes I think using an ES index in association with version should be
able to implement a redis-style list.

In regards to #2 (implementing an atomic counter), according to
Elasticsearch Platform — Find real-time answers at scale | Elastic - this
seems to imply that it is only possible to pass in a REPLACEMENT
version value for the current version value, instead of a positive or
negative # to that be added to the existing version number. Or, is
there a way in which an increment by X (EG: +200, -150, etc...) can be
done?

On Oct 17, 11:12 am, Karussell tableyourt...@googlemail.com wrote:

the third one could be handled with one index (instead of the list)
and using the _version field to avoid problems (can be used to solve
point 2 as well) when adding or deleting items and sort against a
timestamp or similar.

the other mentioned points are probably better implemented with redis
(there are no transactions in ES) ... you could try point 1 with ES:
but even with a ids-request

Elasticsearch Platform — Find real-time answers at scale | Elastic

100k req/sec seems to be high :slight_smile: of course all depends on a lot things
e.g. putting it all into RAM ... let me know how it compares :slight_smile: !

On 17 Okt., 08:08, karmi karel.mina...@gmail.com wrote:

In all three points you mention: GET, INCRBY, atomic LPOP, I'd
consider Redis superior to Elasticsearch.

I wouldn't get rid of Redis just to "simplify the stack". There's
always something where Redis fits as a glove.

(And I love Elasticsearch as much as Redis, possible more :slight_smile:

Karel

On Oct 16, 11:20 pm, datadev n...@adinfocenter.com wrote:

I am currently considering replacing redis with elasticsearch to
simplify the stack and have run into the following 3 use cases:

  1. I am currently using redis for caching. Redis is benchmarked at
    around 100,000 GETs per second when querying by key (http://redis.io/
    topics/benchmarks). How does ES stack in terms simple GET by _id
    performance (assuming real-time get)?
  1. I am currently using redis to store counters (EG: 'foo = foo +
    1280' types of commands). Is there any way to implement the concept of
    counters in ES with support atomic updates (maybe using the script
    command)?
  1. I am currently using redis to store lists for queuing use cases (in
    which items are added to a list and then popped off). Can something
    like that be effectively implemented in ES?

Thanks and keep up the great work!

5

Heya,

Well, elasticsearch will definitely be slower than redis, by how much its
hard to say. Realtime get is pretty fast, but still goes to disk.

Versioning can be used to implement atomic updates. You can get the
document you want, change it, and try and index it back with the version you
got, if it succeeds, all is well, if not, you need to retry. Note though,
that the whole document needs to be reindexed...

On Tue, Oct 18, 2011 at 8:46 AM, datadev nji@adinfocenter.com wrote:

Yes I think using an ES index in association with version should be
able to implement a redis-style list.

In regards to #2 (implementing an atomic counter), according to
Elasticsearch Platform — Find real-time answers at scale | Elastic - this
seems to imply that it is only possible to pass in a REPLACEMENT
version value for the current version value, instead of a positive or
negative # to that be added to the existing version number. Or, is
there a way in which an increment by X (EG: +200, -150, etc...) can be
done?

On Oct 17, 11:12 am, Karussell tableyourt...@googlemail.com wrote:

the third one could be handled with one index (instead of the list)
and using the _version field to avoid problems (can be used to solve
point 2 as well) when adding or deleting items and sort against a
timestamp or similar.

the other mentioned points are probably better implemented with redis
(there are no transactions in ES) ... you could try point 1 with ES:
but even with a ids-request

Elasticsearch Platform — Find real-time answers at scale | Elastic

100k req/sec seems to be high :slight_smile: of course all depends on a lot things
e.g. putting it all into RAM ... let me know how it compares :slight_smile: !

On 17 Okt., 08:08, karmi karel.mina...@gmail.com wrote:

In all three points you mention: GET, INCRBY, atomic LPOP, I'd
consider Redis superior to Elasticsearch.

I wouldn't get rid of Redis just to "simplify the stack". There's
always something where Redis fits as a glove.

(And I love Elasticsearch as much as Redis, possible more :slight_smile:

Karel

On Oct 16, 11:20 pm, datadev n...@adinfocenter.com wrote:

I am currently considering replacing redis with elasticsearch to
simplify the stack and have run into the following 3 use cases:

  1. I am currently using redis for caching. Redis is benchmarked at
    around 100,000 GETs per second when querying by key (
    http://redis.io/
    topics/benchmarks). How does ES stack in terms simple GET by _id
    performance (assuming real-time get)?
  1. I am currently using redis to store counters (EG: 'foo = foo +
    1280' types of commands). Is there any way to implement the concept
    of
    counters in ES with support atomic updates (maybe using the script
    command)?
  1. I am currently using redis to store lists for queuing use cases
    (in
    which items are added to a list and then popped off). Can something
    like that be effectively implemented in ES?

Thanks and keep up the great work!