Client to cluster acces design

Elastic Stack Elasticsearch
1

Hello,

We are wondering about the best design for a client to address an ES
cluster.

  1. The client app knows only an URL of the loadbalancer like haproxy which
    roundrobin request on each nodes (it is the SPOF so you have to handle that)

  2. The client app implement a local elasticsearch beside node without data
    that join the active cluster and the app query the local ES. (maybe 2 or 3
    main nodes have to be knows to jojn the whole cluster)

  3. The client knows a lists of ES node to address via client implementation

    • we have to maintain this list on each app when we upgrade the cluster
    • maybe client can learn from the first accessed node the full list of
      nodes available in the cluster
    • zookeeper ?

Cheers,

--
Damien

2

Hi,

I think you can learn many answers by reading Elasticsearch Platform — Find real-time answers at scale | Elastic

  1. It is possible but to me it seems like extra work that is not necessary (see #2 and #3 as both options are provided by ES out of the box)
  2. This is Node client
  3. This is Transport client

--
Regards,
Lukas

On Tuesday, December 13, 2011 at 4:06 PM, Damien Hardy wrote:

Hello,

We are wondering about the best design for a client to address an ES cluster.

  1. The client app knows only an URL of the loadbalancer like haproxy which roundrobin request on each nodes (it is the SPOF so you have to handle that)

  2. The client app implement a local elasticsearch beside node without data that join the active cluster and the app query the local ES. (maybe 2 or 3 main nodes have to be knows to jojn the whole cluster)

  3. The client knows a lists of ES node to address via client implementation

    • we have to maintain this list on each app when we upgrade the cluster
    • maybe client can learn from the first accessed node the full list of nodes available in the cluster
    • zookeeper ?

Cheers,

--
Damien

3

Lukas answered for the Java client case, but I assume you are talking about
the HTTP based case? I like 3, where the client has a list of URLs (there
is a nice discussion about it on Tire (ruby client)), though 2 is a valid
option as well, start an elasticsearch node that holds no data and is not
an eligible master on your local box, and only talk to it. It will join the
cluster and know about the rest of the nodes.

On Tue, Dec 13, 2011 at 5:06 PM, Damien Hardy damienhardy.bal@gmail.comwrote:

Hello,

We are wondering about the best design for a client to address an ES
cluster.

  1. The client app knows only an URL of the loadbalancer like haproxy which
    roundrobin request on each nodes (it is the SPOF so you have to handle that)

  2. The client app implement a local elasticsearch beside node without data
    that join the active cluster and the app query the local ES. (maybe 2 or 3
    main nodes have to be knows to jojn the whole cluster)

  3. The client knows a lists of ES node to address via client
    implementation

    • we have to maintain this list on each app when we upgrade the
      cluster
    • maybe client can learn from the first accessed node the full list
      of nodes available in the cluster
    • zookeeper ?

Cheers,

--
Damien

4

Hello,

Thank you both for answer,

And yes I mainlly was thinking about non-java centric clients apps :slight_smile:
In the third we have olso to deal with defunct/mising ES nodes that is
managed by a beside private ES data-less node ( solution 2 )

Cheers,

--
Damien

2011/12/13 Shay Banon kimchy@gmail.com

Lukas answered for the Java client case, but I assume you are talking
about the HTTP based case? I like 3, where the client has a list of URLs
(there is a nice discussion about it on Tire (ruby client)), though 2 is a
valid option as well, start an elasticsearch node that holds no data and is
not an eligible master on your local box, and only talk to it. It will join
the cluster and know about the rest of the nodes.

On Tue, Dec 13, 2011 at 5:06 PM, Damien Hardy damienhardy.bal@gmail.comwrote:

Hello,

We are wondering about the best design for a client to address an ES
cluster.

  1. The client app knows only an URL of the loadbalancer like haproxy
    which roundrobin request on each nodes (it is the SPOF so you have to
    handle that)

  2. The client app implement a local elasticsearch beside node without
    data that join the active cluster and the app query the local ES. (maybe 2
    or 3 main nodes have to be knows to jojn the whole cluster)

  3. The client knows a lists of ES node to address via client
    implementation

    • we have to maintain this list on each app when we upgrade the
      cluster
    • maybe client can learn from the first accessed node the full list
      of nodes available in the cluster
    • zookeeper ?

Cheers,

--
Damien